anthony sulistio phd thesis 2008

Download Anthony Sulistio PhD Thesis 2008

Post on 27-Apr-2015

251 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

Advance Reservation and Revenue-based ResourceManagement for Grid SystemsbyAnthony SulistioSubmitted in total fullment ofthe requirements for the degree ofDoctor of PhilosophyMay 2008Department of Computer Science and Software EngineeringThe University of MelbourneAustraliaAdvance Reservation and Revenue-based ResourceManagement for Grid SystemsAnthony SulistioSupervisors: Assoc. Prof. Rajkumar Buyya and Prof. Rao KotagiriAbstractIn most Grid systems, submitted jobs are initially placed into a queue if there areno available compute nodes. Therefore, there is no guarantee as to when these jobs willbe executed. This usage policy may cause a problem for time-critical applications ortask graphs where jobs have inter-dependencies. To address this issue, using advancereservation (AR) in Grid systems would allow users to secure or guarantee resources priorto executing their jobs.This thesis proposes the use of modeling and simulation, since various Grid scenariosneed to be evaluated and repeated. Therefore, this thesis describes the development ofGridSim, a discrete-event Grid simulation tool, which allows modeling and simulation ofvarious properties, such as advance reservation, dierentiated level of network Quality ofService (QoS), data Grid and resource discovery in a virtual organization.This thesis investigates how AR can be incorporated and deployed in Grid systems, anddetermines how to increase the resource utilization. Towards accomplishing these ndings,this thesis presents a system model for scheduling task graphs with advance reservation andinterweaving to increase resource utilization, and proposes a new data structure, namedGrid advance reservation Queue (GarQ), for administering reservations in the Grid systemeciently. In addition, this thesis provides a case for an elastic reservation model, whereusers can self-select or choose the best option in reserving their jobs, according to their QoSneeds, such as deadline and budget. This thesis adapts an on-line strip packing algorithminto the elastic model to reduce the number of rejections and fragmentations (idle timegaps) caused by having reservations in the Grid system.This thesis investigates how to increase resource revenue, and examines how to regulateresource supplies and reservation demands. Towards accomplishing these inquests, thisthesis suggests the use of Revenue Management to determine the pricing of reservations,increase resource revenue, and regulate supply and demand. Moreover, this thesis looksinto overbooking models to protect resources against unexpected cancellations and no-shows of reservations.This is to certify that(i) the thesis comprises only my original work,(ii) due acknowledgement has been made in the text to all other material used,(iii) the thesis is less than 100,000 words in length, exclusive of table, maps, bibliogra-phies, appendices and footnotes.SignatureDateAcknowledgmentsFirst of all, I would like to thank my principal supervisor Assoc. Prof. Rajkumar Buyyafor his advice, encouragement and guidance throughout my candidature. In addition, Iwould like to thank my co-supervisor Prof. Rao Kotagiri for his comments and remarks.Your expertise and knowledge have inuenced the direction of my research.I am also grateful to the following people: Gokul Poduval and Assoc. Prof. Chen-Khong Tham (National University of Singapore), Prof. Dr. Wolfram Schimann (Univer-sity of Hagen, Germany), Dr. Uros Cibej and Prof. Borut Robic (University of Ljubljana,Slovenia), Prof. Sushil Prasad (Georgia State University, USA), Agustin Caminero, Dr.Blanca Caminero, and Assoc. Prof. Carmen Carrion (Universidad de Castilla-La Mancha,Spain), and Dr. Kyong Hoon Kim (Gyeongsang National University, Korea). It has beena pleasure exchanging ideas and working with all of you.I would like to thank Assoc. Prof. Henri Casanova (University of Hawaii at Manoa,USA) for his excellent comments on my PhD conrmation report, and Dr. Udo Hoenig(University of Hagen, Germany) for giving me the access and technical support for thetest bench structure used in my thesis. Moreover, I want to express my gratitude to Dr.Anirban Chakrabarti (Infosys Technologies, Bangalore, India), and external examiners ofthis thesis for their constructive comments.I would also like to thank to all the past and current members of the GRIDS Lab,University of Melbourne. In particular, Dr. Srikumar Venugopal, Dr. Tianchi Ma, Dr.James Broberg, and Chee Shin Yeo for their help and constructive comments. My gratitudealso extend to the Departments administrative and IT sta: Dr. James Bailey, PinooBharucha, Cindy Sexton, Adam Hendrix, Michael Poloni, Binh Phan, and Julien Reid fortheir help.Special thanks to Denzil Andrews and Donny Poh for their support and understanding.In addition, thanks to Xulio L. Albin, Jia Yu, Hussein Gibbins, Krishna Nadiminti, Prof.Dr. Christoph Reich, Matthias Banholzer, and Dr. Yoshitake Kobayashi for their companyvin recreational sports and social events, which made my study more enjoyable and lessstressful.Finally, I would like to thank my family for their love, support and help in every aspectof life. I could not have done it without you.The work presented in this thesis was partially supported by research grants from theAustralian Research Council (ARC) and Australian Department of Education, Scienceand Training (DEST).Anthony SulistioMelbourne, AustraliaMay 2008ContentsList of Figures xiiList of Tables xiiiList of Algorithms xvList of Frequently Used Acronyms and Notations xviii1 Introduction 11.1 Grid Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.2.1 The Need for Advance Reservation . . . . . . . . . . . . . . . . . . . 41.2.2 The Importance of Economy Model . . . . . . . . . . . . . . . . . . 61.2.3 A Case for Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . 71.3 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.4 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 Related Work on Advance Reservation Projects in Networks & Grids 132.1 Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.1.1 On-Demand Secure Circuits and Advance Reservation System . . . . 142.1.2 Internet2 Bandwidth Reservation for User Work . . . . . . . . . . . 152.1.3 GEANT2 Advance Multi-domain Provisioning System . . . . . . . . 152.2 Grids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.2.1 Maui Scheduler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.2.2 Dynamic Soft Real-Time (DSRT) Scheduling System . . . . . . . . . 182.2.3 PBS Pro . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.2.4 Sun Grid Engine (SGE) . . . . . . . . . . . . . . . . . . . . . . . . . 202.2.5 Globus Architecture for Reservation and Allocation (GARA) . . . . 212.2.6 Highly-Available Resource Co-Allocator (HARC) . . . . . . . . . . . 212.2.7 G-lambda Grid Scheduling System . . . . . . . . . . . . . . . . . . . 222.2.8 Grid Capacity Planning . . . . . . . . . . . . . . . . . . . . . . . . . 232.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243 A Grid Simulator that Supports Advance Reservation 253.1 Grid Simulation Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.2 GridSim Toolkit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.2.1 GridSim Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . 283.2.2 Fundamental Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . 29vii3.2.3 New GridSim Design . . . . . . . . . . . . . . . . . . . . . . . . . . . 313.3 Design and Implementation of Advance Reservation . . . . . . . . . . . . . 373.3.1 States of Advance Reservation . . . . . . . . . . . . . . . . . . . . . 373.3.2 Extensible Grid Resource Framework . . . . . . . . . . . . . . . . . . 393.3.3 GridSim Application Programming Interface . . . . . . .