![Page 1: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/1.jpg)
Resource Selector
Chuang Liu
![Page 2: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/2.jpg)
What do we want to do?
• A smart Resource Selector
App
R S
Resource requirement
![Page 3: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/3.jpg)
What do we want to do?
App
R SThese Resources seem fit your requirement best
![Page 4: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/4.jpg)
Goal
• A Resource Selector for general purpose• Matching between application’s requirement and a
set of resources• Adaptability to dynamic status of distributed
environment• Support ownership of resource • Support performance model of application• The interface between application and RS should
be simple
![Page 5: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/5.jpg)
RS--Structure
Resource
Selector GRIS
GIISMDSAp
p
RS
MatchMaker
RequirementResource
A subset of resource
App
![Page 6: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/6.jpg)
Challenges
• How to specify resource and request and how to match request with resource.
• Consistency of data in RS and system status.
• How to choose N resources from M available resources. (N <=M)
![Page 7: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/7.jpg)
Challenges
• How to specify the request and resource and how to match the request with resource
• Consistency of data in RS and system status.
• How to choose N resources from M available resources. (N <=M)
![Page 8: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/8.jpg)
ClassAd—Mechanism to specify resources and request
• The classad mechanism is a language for expressing and evaluating attribute
• A classad is a set of named expressions
• Each named expression is called an attribute
• Expression similar to those found in C/C++
![Page 9: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/9.jpg)
ClassAd : example(resource)
• [• OpSys=“LINUX”;• Name= “trapezius.cs.uchicago.edu”;• LoadAvg= 0.03;• Friends = (“foster”, “dave”);• Untrusted = (“evil”, “rival”);• Constraints= !member(other.Owner, Untrusted) &&
(LoadAvg < 0.3);• ]
![Page 10: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/10.jpg)
ClassAd : example(request)
• [
• Owner=“chliu”;
• Requirements= other.LoadAvg < 0.3 && other.opSys=“LINUX”;
• Rank = 1/other.LoadAvg;
• ]
![Page 11: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/11.jpg)
Challenges
• How to specify the resource and request and how to match the request with resource.
• Consistency of data in RS and system status.
• How to choose N resources from M available resources. (N <=M)
![Page 12: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/12.jpg)
Consistency
• Several threads in RS update information about system status based TTL value.– Update information about available resource by
access GIIS– Update information about status of every
resource by access GRIS or GIIS• Tradeoff between Performance and Consistency
![Page 13: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/13.jpg)
Challenges
• How to specify the resource and request and how to match the request with resource.
• Consistency of data in RS and system status.
• How to choose N resources from M available resources which fit application’s requirement best. (N <=M)
![Page 14: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/14.jpg)
Resource Selection
• How to select N resource from M available resources efficiently.
• How to judge which one is best among several matched results
![Page 15: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/15.jpg)
Resource Selection
• How to select N resource from M available resources efficiently.
• How to judge which one is best among several matched results
![Page 16: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/16.jpg)
Criteria to judge the desirability of resource
• Performance model– F(resource Info, application info)
– [ minCPUSpeed > 10 MIPS
– minMemSize > 100 MB
– Rank= minCPUSpeed * NumOfResource
– ]
– Embed a program or function call in Classad• ? Classad don’t support function call in expression.
![Page 17: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/17.jpg)
Resource Selection
• How to Select N resources from M available resources
• How to judge which one is best among several matched results
![Page 18: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/18.jpg)
Bilateral match- Clique
• Organize all available resources into several cliques.
Classads for resource
Classads for clique
Classads for requirement
Match maker
![Page 19: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/19.jpg)
Clique- How to organize clique
Methods to organize clique– Manually– Automatically– Pros:
• Easy and useful
– Cons:• Not flexibility
![Page 20: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/20.jpg)
Clique- Naive
• Naive method– For example:
• Resource: { a, b, c}
• Cliques {a}, {b}, {c}, {a, b}, {b,c}, {a,c}, {a, b, c}
– Cons• The number of clique is 2 to N
![Page 21: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/21.jpg)
Gang Match
• Gang Matching 1 N
Classads for resource
Classads for requirement
Match maker
![Page 22: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/22.jpg)
Gang match- Greedy Algorithm
• Greedy Algorithm– Clique, candidate = null– Match the requirement with every resource– Choose resource with highest ranking as the first number of clique– For(;;) {– If (clique match requirement) and (performance of application increase)– Candidate=clique– Match the requirement with resource which is not in clique– Add node with highest ranking in the left nodes to clique– Else– Return Candidate – }
![Page 23: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/23.jpg)
Gang match- Greedy Algorithm
• Pros:– High performance– Give pretty good optimal result to loosely
coupled application
• Cons:– Locally optimal choice does not always lead to
globally optimal solution
![Page 24: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/24.jpg)
Gang match- Port and docking
• Ports and docking• [ Ports = {• [ Label = Host1;• Requirements= MemorySize > 17.2M;• Constraint = Host1.Arch == "INTEL" && Host1.OpSys == "LINUX";• Rank = Host1.MIPS• ],• [ Label = Host2;• Requirements= MemoryReqs > 18 M;• Constraint = Host2.Arch == "INTEL" && Host2.OpSys == "LINUX"
&&• Host1.Subnet == Host2.Subnet• Rank = Host2.KFlops• ]• Rank = 1 * Host1.MIPS + 8 * Host2.Kflops;• } ]
![Page 25: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/25.jpg)
Gang match- Port and docking
• Pros:– Internal mechanism provided by Classad– ? Available in Classad package
• Cons:– Match performance– Application need to specify how many nodes it
wanted
![Page 26: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/26.jpg)
Gang match- Dynamic programming
• Limited resource use– System administrator control how many
resources user can use– User’s requirement
• [ performance > threadhold value && number of resource is as little as possible]
– Application’s requirement • App specify how many CPU it wanted in request• A 4 X 4 X 4 Grid calculation, numOfCPU < 64
![Page 27: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/27.jpg)
Gang match Dynamic programming
• Multi-dimensions knapsack problem– Knapsack problem : “There are M items, every
item has a Weight and a Value. Try to choose items from these items such that their value is maximum and their total weight is less than W.”
![Page 28: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/28.jpg)
Gang match- Dynamic programming
• Pros:– More flexible– Always provide good resource to application– polynomial time algorithm o(N)
• Cons:– Performance is bad than greedy algorithm
![Page 29: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/29.jpg)
Project Status
• A RS is running, but not smart enough.– Two match strateges have been implemented
• Clique
• Greedy algorithm
![Page 30: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/30.jpg)
Open problem
• How well does RS work?– Performance of application on selected resource– Cost of RS
• Evaluate different strategy in match making
![Page 31: Resource Selector Chuang Liu. What do we want to do? A smart Resource Selector App R S Resource requirement](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d625503460f94a43df7/html5/thumbnails/31.jpg)
Welcome comment and suggestion
Thank you!