Economy Grid: A New e-Paradigm for Grid/Internet Computing
GAURAV GUNJAN JHAGAURAV GUNJAN JHA Email:- [email protected]:- [email protected]
BTECH/COMP.SCIENCEBTECH/COMP.SCIENCEBIT MESRA,BIT MESRA,INDIAINDIA
What is Grid ?What is Grid ?An infrastructure that couples:An infrastructure that couples:
– Computers (PCs, workstations, clusters, traditional Computers (PCs, workstations, clusters, traditional supercomputers, and even laptops, notebooks, mobile computers, supercomputers, and even laptops, notebooks, mobile computers, PDA, and so on) …PDA, and so on) …
– Software ? (e.g., ASPs renting expensive special purpose Software ? (e.g., ASPs renting expensive special purpose applications on demand)applications on demand)
– Catalogued Data/Databases (e.g., transparent access to human Catalogued Data/Databases (e.g., transparent access to human genome database)genome database)
– Special Instruments (e.g., radio telescope--SETI@Home Searching Special Instruments (e.g., radio telescope--SETI@Home Searching for Life in galaxy, Austrophysics@Swinburne for pulsars)for Life in galaxy, Austrophysics@Swinburne for pulsars)
– People/collaborators (even animals who knows ?)People/collaborators (even animals who knows ?)
and offers a simple, consistent, dependable, & pervasive and offers a simple, consistent, dependable, & pervasive access across (local/wide-area) networks to present them access across (local/wide-area) networks to present them as an unified integrated resource.as an unified integrated resource.
Who pays for all this ?Who pays for all this ?Any Incentive for GRID resource Any Incentive for GRID resource owners ?owners ?
GUSTO
Distributed ASCI Supercomputer NASA IPG
Building ofBuilding ofa “brokerage” system…..a “brokerage” system…..
Foundation for the Grid Economy
Economy Grid: GRACEEconomy Grid: GRACE
GrGrid id AArchitecture for rchitecture for CComputational omputational EEconomyconomy
GRACE aims help Nimrod/G overcome the current limitations.GRACE aims help Nimrod/G overcome the current limitations.
GRACE middleware offer generic interfaces (APIs) that other developers of grid tools can use along with GRACE middleware offer generic interfaces (APIs) that other developers of grid tools can use along with Globus services.Globus services.
Why Computational Economy in Why Computational Economy in Resource Management ?Resource Management ?
““Observe Grid characteristics and current resource management Observe Grid characteristics and current resource management policies”policies”
Grid resources are not owned by user or single organisation.Grid resources are not owned by user or single organisation.
They have their own administrative policy They have their own administrative policy
Mismatch in resource demand and supplyMismatch in resource demand and supply– overall resource demand may exceed supply.overall resource demand may exceed supply.
Markets are an effective institution in coordinating the activities of Markets are an effective institution in coordinating the activities of several entities.several entities.
Traditional System-centric (performance matrix approaches does not Traditional System-centric (performance matrix approaches does not suit in grid environment.suit in grid environment.
– System-Centric --> User CentricSystem-Centric --> User Centric
Like in real life, economic-based approach is one of the best ways to Like in real life, economic-based approach is one of the best ways to regulate selection and scheduling on the grid as it captures user-intent.regulate selection and scheduling on the grid as it captures user-intent.
Advantages of Economic-based RMAdvantages of Economic-based RM
Helps in regulating demand and supplyHelps in regulating demand and supply– resource access cost can fluctuate (based on demand resource access cost can fluctuate (based on demand
and supply and system can adapt)and supply and system can adapt)
Scalable SolutionScalable Solution– No need of central coordinator (during negotiation)No need of central coordinator (during negotiation)– Resources(sellers) and also Users(buyers) can make Resources(sellers) and also Users(buyers) can make
their own decisions and try to maximize utility and profit.their own decisions and try to maximize utility and profit.
Uniform Treatment of all ResourcesUniform Treatment of all Resources– Everything can can be traded including CPU, Mem, Net, Everything can can be traded including CPU, Mem, Net,
Storage/Disk, other devices/instrumentsStorage/Disk, other devices/instruments– Efficient allocation of resourcesEfficient allocation of resources
Grid Node N
Grid Node 2
Computational Market Model for Grid Resource ManagementComputational Market Model for Grid Resource Management
Grid User
Application
Grid Resource Broker
Grid Resource/Control Domains
Grid Explorer
Schedule Advisor
Trade Manager
Job ControlAgent
Deployment Agent
Trade Server
Resource Allocation
ResourceReservation
R1
Other services
Grid Information Server(s)
R2 Rm…
Charging Alg.
Accounting
Grid Node1
…
Trading
Grid Middleware
…
Info ?
…
Jobs
HealthMonitor
Grid Open Trading ProtocolsGrid Open Trading Protocols
Get Connected
Call for Bid(DT)
Reply to Bid (DT)
Negotiate Deal(DT)
Confirm Deal(DT, Y/N)
….
Cancel Deal(DT)
Change Deal(DT)
Get Disconnected
Trade Manager Trade Server
Pricing Rules
DT - Deal Template - resource requirements (BM) - resource profile (BS) - price (any one can set) - status - change the above values - negotiation can continue - accept/decline - validity period
API
Economic Models for ResourceTrading
l Commodity Market Model
l Posted Prices Models
l Tendering Model
l Auction Model
l Shareholder Model
l Partnership Model
A Commodity Market Model
“S olve this in5hrs for $20”
Grid Market Directory (GMD)
ResourceBroker
Grid I nfo. Service
GTS
GTS
(Grid Service Provider)
GTS
GTS GTS
“register me as GSP”
“Give me list of GSPs”
(GTS - Grid Trade Server)
(GSP)
(RB selects GSPs)
Posted Price Model
“Solve this bynext day for $5”
Grid Market Directory (GMD)
ResourceBroker
“2hrs SP2, $5”
Grid I nfo. Service
GTS
GTS
(Grid Service Provider)
GTS
GTS GTS
“T3E, $9/ hr, Sunday”
“Free for Genome”“10% discount today”
“Any SP2/ T3E? offers”“Free or < $2/ hr clusters+matlab”
“5MB free”
(GTS - Grid Trade Server)
(GSP)
Tender Model
“Solve this in15hrs for $10”
Grid Market Directory (GMD)
ResourceBroker
Grid I nfo. Service
GTS
GTS
(Grid Service Provider)
GTS
GTS GTS
“Any Ads for service tenders”
“Post: call for tenders”
(GTS - Grid Trade Server)
(GSP)
(GSPs bid)
Is it Dutch Auction ?Buyers name theirprice and suppliescompete to bid thelowest price.Eg: GotFrom .com
Auction Model
Grid Market Auctioneer (GMA)
ResourceBroker
“SP2 time, 9pm-8am”
Grid I nfo. Service
GTS
GTS
(Grid Service Provider)
GTS
GTS GTS
“Post: auction T3E service”
“Solve this in20 hrs for $5”
(GTS - Grid Trade Server)
(GSP)
ResourceBroker
….
“Solve this in1 hrs for $35”
(RBs bid)
English Auction ? (ebay.com)(Sellers name their priceand) buyers compete to bidthe highest price.Vickr ey Auction ?: Highestbidder gets at the price ofsecond highest bidder!!
Service I tems to be Charged
l CPU - User and System time
l Memory:
u maximum resident set size - page size
u amount of memory used
u page faults: with/without physical I /O
l Storage: size, r/ w/ block I O operations
l Network: msgs sent/ received
l Signals received, context switches
l Software and Libraries accessed
How to decide Price ?
l Fixed price model ( like today’s Internet)
l Dynamic ( like tomorrow’s I nternet)
l Demand and Supply
l Usage Period
l Loyalty of Customers ( like Airlines favoringfrequently flyers!)
l Historical data
l Advance Agreement (like Monash getting 17%discount for flights)
l Usage Timing (peak, off-peak, lunch time)
l Calendar based (holiday/vacation period)
l Bulk Purchase (register 100 .com domains at once!)
l Resource capability as benchmarked in the market!
Accounting, Allocations and J obScheduling Flow @ each site/Grid Level
QBankQBank
Maui SchedulerMaui Scheduler44
IBM-LL/PBS/….IBM-LL/PBS/….
00
55 88
66 77
Compute ResourcesCompute Resourcesclusters/SGI/SP/...clusters/SGI/SP/...
0. Make Depo sits, Transfer s, Refunds, Que ries/Reports1. Clients neg otia tes for acce ss cost.2. Negotiation is performed per owner defined policies.3. If cli ent is h appy, TS in forms QB abo ut acce ss d eal.4. Job is Submitte d5. Check with QB for “go ahead”6. Job Sta rts7. Job Comple tes8. Infor m QB abou t resource resource uti liza tion .
TradeServerTradeServer 3311
ChargingAlgChargingAlg..22
DBDB@Each Site@Each Site
GRID BankGRID Bank
Payment
l Buy credits in advance / GSPs bill the userlater- -”pay as you go”
l Pay by Electronic Currency
u NetCash (anonymity) and NetChequel http:/ / www. isi.edu/ gost/ info/ netcash/
l http:/ / www. isi.edu/ gost/ info/ netcheque/
u NetCheque:l Users register with NC accounting servers, can write
electronic cheques and send (e.g email). Whendeposited, balance is transferred from sender toreceiver account.
u NetCashl I t supports anonymity and it uses the NetCheque
system to clear payments between currency servers.
Nimrod/G Resource BrokerNimrod/G Resource Broker
Nimrod/G Approach to Resource Nimrod/G Approach to Resource Management and SchedulingManagement and Scheduling
A global scheduler for managing and steering task A global scheduler for managing and steering task farming (parametric simulation) applications on farming (parametric simulation) applications on computational grid based on computational grid based on deadlinedeadline and and computational economycomputational economy. .
Key FeaturesKey Features– A single window to manage & control experimentA single window to manage & control experiment– Resource DiscoveryResource Discovery– Trade for ResourcesTrade for Resources– SchedulingScheduling– Steering & data managementSteering & data management
It allows to study the behaviour of some of the output It allows to study the behaviour of some of the output variables against a range of different input scenarios.variables against a range of different input scenarios.
What is Nimrod/G ?What is Nimrod/G ?
A Nimrod/G ClientA Nimrod/G Client
CostCostDeadlineDeadline
Legion hosts
Globus Hosts
Bezek is in both Globus and Legion Domains
Arlington
Alexandria
Richmond
HamptonNorfolk
Virginia BeachChesapeakePortsmouth
Newport News
Roanoke
Ap p om a toxRiver
Ja m esRiver
Shena nd oa hRiver
Ra p p a ha nnoc kRiver
Potom a cRiver
VIRGINIA77
81
64
64
66
85
Conclude with a comparison with Conclude with a comparison with the Electrical Grid………..the Electrical Grid………..
Where we Where we are ????are ????
Alessandro VoltaAlessandro Volta in Paris inin Paris in 18011801 inside France National inside France National Institute shows the battery at the presence ofInstitute shows the battery at the presence of Napoleon INapoleon I
Fresco by N. Cianfanelli (1841) Fresco by N. Cianfanelli (1841)
(Zoological Section "La Specula" of National History Museum of Florence University)(Zoological Section "La Specula" of National History Museum of Florence University)
….and in the future,
I imagine a worldwide
Power (Electrical) Grid …...
What ?!?!This is a mad man…
Oh, monDieu !
2000 - 1801 =2000 - 1801 = 199 Years199 Years
What will be the dominant grid approach in the next future ??
TrendsTrends
It is very difficult to predict the future and this is particular true in a field such as
Information Technology
“I think there is a world market for about five computers.”Thomas J. Watson Sr., IBM Founder, 1943
TrendsTrends
The time is exciting but the way is hard and The time is exciting but the way is hard and long….long….
GRID
ConclusionsConclusionsThe Emergence of Internet as a Powerful connectivity media is bridging the gap between a number of technologies leading to what is known as “Everything on IP”.The Emergence of Internet as a Powerful connectivity media is bridging the gap between a number of technologies leading to what is known as “Everything on IP”.Cluster-based systems have become a platform of choice for mainstream computing. Cluster-based systems have become a platform of choice for mainstream computing. A number of GRID project world-wide have been presented to explore computing trend!A number of GRID project world-wide have been presented to explore computing trend!Economic based approach to resource management is the way to go in the grid environment.Economic based approach to resource management is the way to go in the grid environment.Both sequential and parallel applications run seamless on desktops, SMPs, Clusters, and the Grid without any change.Both sequential and parallel applications run seamless on desktops, SMPs, Clusters, and the Grid without any change.Grid: A Next Generation Internet ?Grid: A Next Generation Internet ?
Thank You For giving me
A patient hearing !!!!
Thank You For giving me
A patient hearing !!!!
