Grid ComputingGrid ComputingIts Promise and ChallengesIts Promise and Challenges

Tom SmithTom SmithMaster’s CandidateMaster’s CandidateComputer ScienceComputer Science

Union CollegeUnion CollegeJanuary 2004January 2004

What is Grid Computing?What is Grid Computing?

The word “grid” is chosen by analogy with The word “grid” is chosen by analogy with the power grid:the power grid: ““always on” resourcealways on” resource Users don’t need to know where the resource Users don’t need to know where the resource

is coming fromis coming from Global (or at least national) standardGlobal (or at least national) standard

The Grid may be as revolutionary as the The Grid may be as revolutionary as the power grid and the PC were at their time power grid and the PC were at their time of introduction.of introduction.

What is Grid computing?What is Grid computing?

Classic example: SETI@HomeClassic example: SETI@Home The basic definition (for today): Grid The basic definition (for today): Grid

computing is assembling more than computing is assembling more than machine or system into a unified resource.machine or system into a unified resource.

Little agreement about what exactly Little agreement about what exactly constitutes a “grid”constitutes a “grid”

Refer to “progression of parallel Refer to “progression of parallel computing”computing”

What is Grid computing?What is Grid computing?

Ian Foster: Luminous figure in the field, Ian Foster: Luminous figure in the field, based at U of Chicago and Argonne based at U of Chicago and Argonne National LaboratoryNational Laboratory

Argues for a formal definition of the gridArgues for a formal definition of the grid

Foster’s DefinitionFoster’s Definition

Grids are motivated by Grids are motivated by virtual virtual organizationsorganizations

Grids are heterogeneous in many ways:Grids are heterogeneous in many ways: Diverse machine configurationsDiverse machine configurations Diverse network issuesDiverse network issues Diverse authentication methodsDiverse authentication methods

““Real” grids embrace heterogeneityReal” grids embrace heterogeneity

Foster’s Definition, cont’dFoster’s Definition, cont’d

Grids should be able to talk to one another Grids should be able to talk to one another (intergrid protocols)(intergrid protocols)

Most important: grids allow access to a variety of Most important: grids allow access to a variety of resources (not just cycles)resources (not just cycles) SensorsSensors StorageStorage DatabasesDatabases MemoryMemory ApplicationsApplications Etc.Etc.

Early Success: SETI@HomeEarly Success: SETI@Home

Search for ExtraTerrestrial IntelligenceSearch for ExtraTerrestrial Intelligence Runs as a screensaver/background appRuns as a screensaver/background app 35GB daily tapes split into 250kb chunks35GB daily tapes split into 250kb chunks Almost 5 million usersAlmost 5 million users ~1500 signed up yesterday~1500 signed up yesterday 1,781,106.179 1,781,106.179 yearsyears of CPU time has of CPU time has

been contributed to datebeen contributed to date

Early Success: United DevicesEarly Success: United Devices

Based on intranet at Novartis (drug Based on intranet at Novartis (drug research)research)

Molecular modelingMolecular modeling Alternative to buying a new high-Alternative to buying a new high-

performance computerperformance computer Saved $2,000,000 with a $400,000 Saved $2,000,000 with a $400,000

investmentinvestment

United Devices: cont’dUnited Devices: cont’d

Projects that would have taken 6 years, Projects that would have taken 6 years, now take 12 hours.now take 12 hours.

Initial test was on 2,700 corporate PCsInitial test was on 2,700 corporate PCs So successful, installing it on all 70,000 So successful, installing it on all 70,000

corporate desktopscorporate desktops

United Devices: Also availableUnited Devices: Also available

Grid MP Alliance: sell your corporation’s Grid MP Alliance: sell your corporation’s excess cycles to UDexcess cycles to UD

Grid MP Global: “rent” cycles from UDGrid MP Global: “rent” cycles from UD Implementation of something that seemed Implementation of something that seemed

only theoretical short time agoonly theoretical short time ago ““Computing on demand”Computing on demand”

Challenges of Grid computingChallenges of Grid computing

SecuritySecurity Bandwidth/networkingBandwidth/networking Authentication/authorizationAuthentication/authorization StorageStorage Reliability: what if a node goes down?Reliability: what if a node goes down? ReportingReporting SynchronizationSynchronization Nearly every aspect of computer architecture Nearly every aspect of computer architecture

touched.touched.

Strategies for Simplifying ProblemStrategies for Simplifying Problem

Virtualization/abstractionVirtualization/abstraction Virtual OSVirtual OS StandardsStandards

Anatomy of the GridAnatomy of the Grid

Influential paper, 2001: Foster et al.Influential paper, 2001: Foster et al. Introduced notion of virtual organizationIntroduced notion of virtual organization Suggested “an open and extensible grid Suggested “an open and extensible grid

architecture”architecture”

Physiology of the GridPhysiology of the Grid

Another influential paperAnother influential paper Foster et al., June 2002Foster et al., June 2002 New concepts: New concepts:

Open Grid Services Architecture (OGSA)Open Grid Services Architecture (OGSA) Shift in focus from “resources” to “grid Shift in focus from “resources” to “grid

services”services” Grid services are a special instance of web Grid services are a special instance of web

servicesservices

Web ServicesWeb Services

Cross-platform way of letting different Cross-platform way of letting different systems interact and exchange systems interact and exchange informationinformation

Based on WSDL and SOAPBased on WSDL and SOAP

SOAPSOAP

Simple Object Access ProtocolSimple Object Access Protocol Maintained by W3C, current ver. 1.2Maintained by W3C, current ver. 1.2 XML-based, runs on top of HTTPXML-based, runs on top of HTTP Simple, platform-independent way of Simple, platform-independent way of

exchanging info between 2 endpointsexchanging info between 2 endpoints An “envelope” protocolAn “envelope” protocol Click here for exampleClick here for example

WSDLWSDL

Web Services Description LanguageWeb Services Description Language Developed/Supported by IBM, MicrosoftDeveloped/Supported by IBM, Microsoft Proposed standard to W3C, latest proposal is Proposed standard to W3C, latest proposal is

version 1.2version 1.2 XML-based, works in conjunction with SOAP, XML-based, works in conjunction with SOAP,

HTTP, MIMEHTTP, MIME portType: defines function library or class that is portType: defines function library or class that is

being used/requestedbeing used/requested Click here for exampleClick here for example Click here for Complete WSDL example w/SOAClick here for Complete WSDL example w/SOA

P bindingP binding

OGSA and OGSIOGSA and OGSI

OGSA = Open Grid Services ArchitectureOGSA = Open Grid Services Architecture Web Services + standard grid interfaces = Web Services + standard grid interfaces =

grid servicesgrid services OGSI = Open Grid Services InfrastructureOGSI = Open Grid Services Infrastructure OGSI v1.0 was released by Global Grid OGSI v1.0 was released by Global Grid

Forum in June 2003Forum in June 2003 Click here for illustration of OGSAClick here for illustration of OGSA Click here for illustration of OGSIClick here for illustration of OGSI

Note: Globus ToolkitNote: Globus Toolkit

Open-source collection of libraries and Open-source collection of libraries and functions for implementing gridsfunctions for implementing grids

““De facto standard” for grid softwareDe facto standard” for grid software Foster involved from the beginningFoster involved from the beginning Generated much of OGSA, OGSIGenerated much of OGSA, OGSI

Adoption of OGSA/OGSIAdoption of OGSA/OGSI

Initially, confusion reigned (January 2003)Initially, confusion reigned (January 2003) Situation clearer after release of OGSI Situation clearer after release of OGSI

(June 2003)(June 2003) Most of the grid vendors pay at least lip Most of the grid vendors pay at least lip

service to OGSA/OGSIservice to OGSA/OGSI WSDL’s full incorporation of grid concepts WSDL’s full incorporation of grid concepts

may make OGSA/OGSI less importantmay make OGSA/OGSI less important For example: United Devices’ software For example: United Devices’ software

makes use of SOAP and WSDLmakes use of SOAP and WSDL

““The” GridThe” Grid

Akin to “The internet”Akin to “The internet” Still just a conceptStill just a concept No grids talking to each other in any No grids talking to each other in any

significant ways yetsignificant ways yet Lots of obstacles to overcome firstLots of obstacles to overcome first

Case Study: GridFTPCase Study: GridFTP

GridFTP was a protocol developed by GridFTP was a protocol developed by Foster et al. for Globus Toolkit 2.Foster et al. for Globus Toolkit 2.

Click here for imageClick here for image..

Motivations for Grid ComputingMotivations for Grid Computing

High-performance computingHigh-performance computing Processor cycle recoveryProcessor cycle recovery Resource sharingResource sharing CollaborationCollaboration Approaches have been parallel so far, but Approaches have been parallel so far, but

are converging.are converging.

Questions?Questions?

QuestionQuestion

Gregory Andrews: “Even most personal Gregory Andrews: “Even most personal computers will soon have a few computers will soon have a few processors” (2000)processors” (2000)

Will the grid make multiprocessor Will the grid make multiprocessor computers irrelevant?computers irrelevant?

QuestionQuestion

Will “The Grid” become as pervasive as Will “The Grid” become as pervasive as the internet? the internet?

QuestionQuestion

What about bandwidth issues?What about bandwidth issues? How will the irregular rollout of high-speed How will the irregular rollout of high-speed

networks affect the expansion of the grid?networks affect the expansion of the grid?

QuestionQuestion

Will home PCs play any significant role in Will home PCs play any significant role in high-performance computing?high-performance computing?

Will organizations without the need for Will organizations without the need for high-performance computing rent out their high-performance computing rent out their resources?resources?

How will the grid change the nature of the How will the grid change the nature of the PC?PC?

QuestionQuestion

According to According to The New York TimesThe New York Times, Europe , Europe is exceeding the U.S. in their is exceeding the U.S. in their implementation of grid technologies.implementation of grid technologies.

Is this important?Is this important?