Grid Computing and the Open Grid Service Architecture

Download Grid Computing and the Open Grid Service Architecture

Post on 05-Feb-2016

32 views

Category:

Documents

0 download

Embed Size (px)

DESCRIPTION

Grid Computing and the Open Grid Service Architecture. Ian Foster Argonne National Laboratory University of Chicago http://www.mcs.anl.gov/~foster. 2nd IEEE Intl Symp. on Network Computing & Applications, Boston, April 17, 2003. Partial Acknowledgements. - PowerPoint PPT Presentation

TRANSCRIPT

  • Grid Computing and theOpen Grid Service ArchitectureIan Foster

    Argonne National LaboratoryUniversity of Chicago

    http://www.mcs.anl.gov/~foster2nd IEEE Intl Symp. on Network Computing & Applications, Boston, April 17, 2003

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    Abstract In both e-business and e-science, we often need to integrate services across distributed, heterogeneous, dynamic "virtual organizations formed from the disparate resources within a single enterprise and/or via external resource sharing relationships. This integration can be technically challenging due to the need to achieve various qualities of service in heterogeneous environments. I introduce this "Grid opportunity," discuss the origins and applications of Grid technologies in the world of science, and present recent work on an Open Grid Services Architecture that seeks to generalize Grid computing concepts to create a powerful framework for distributed resource sharing and management.

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    Partial AcknowledgementsOpen Grid Services Architecture designCarl Kesselman, Karl Czajkowski @ USC/ISISteve Tuecke @ANLJeff Nick, Steve Graham, Jeff Frey @ IBMGrid services collaborators at ANLKate Keahey, Gregor von LaszewskiThomas Sandholm, Jarek Gawor, John BresnahanGlobus Toolkit R&D also involves many fine scientists & engineers at ANL, USC/ISI, and elsewhere (see www.globus.org)Strong links with many EU, UK, US Grid projectsSupport from DOE, NASA, NSF, IBM, Microsoft

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    OverviewGrid: why and whatEvolution of Grid technologyOpen Grid Services ArchitectureFuture directionsTowards lightweight VOs: dynamic trust relationshipsTowards global knowledge communities: virtual data and dynamic workspaces

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    Why the Grid?(1) Revolution in SciencePre-InternetTheorize &/or experiment, alone or in small teams; publish paperPost-InternetConstruct and mine large databases of observational or simulation dataDevelop simulations & analysesAccess specialized devices remotelyExchange information within distributed multidisciplinary teams

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    Why the Grid?(2) Revolution in BusinessPre-InternetCentral data processing facilityPost-InternetEnterprise computing is highly distributed, heterogeneous, inter-enterprise (B2B)Business processes increasingly computing- & data-richOutsourcing becomes feasible => service providers of various sorts

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    New OpportunitiesDemand New Technology Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations

    When the network is as fast as the computer's internal links, the machine disintegrates across the net into a set of special purpose appliances (George Gilder)

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    Grid Communities & TechnologiesYesterdaySmall, static communities, primarily in scienceFocus on sharing of computing resourcesGlobus Toolkit as technology baseTodayLarger communities in science; early industryFocused on sharing of data and computingOpen Grid Services Architecture emerging TomorrowLarge, dynamic, diverse communities that share a wide variety of services, resources, dataNew issues: Trust, distributed RM, knowledge

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    NSF TeraGridNCSA, SDSC, Argonne, CaltechUnprecedented capability13.6 trillion flop/s600 terabytes of data40 gigabits per secondAccessible to thousands of scientists working on advanced researchwww.teragrid.org

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    Data Grids for High Energy PhysicsEnable international community of 1000s to access & analyze petabytes of dataHarness computing & storage worldwideVirtual data concepts: manage programs, data, workflowDistributed system management

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    NEESgrid Earthquake Engineering CollaboratoryU.Nevada Renowww.neesgrid.org

    Grid, Globus Toolkit, and OGSA

  • Network for Earthquake Engineering Simulation

    Field Equipment

    Laboratory Equipment

    Remote Users

    Remote Users: (K-12 Faculty and Students)

    High-Performance Network(s)

    Instrumented Structures and Sites

    Leading Edge Computation

    Curated Data Repository

    Laboratory Equipment (Faculty and Students)

    Global Connections(fully developed FY 2005 FY 2014)

    (Faculty, Students, Practitioners)

    12/06/01 MRE Panel

    *foster@mcs.anl.govARGONNE CHICAGO

    Grid ComputingGrid ComputingBy M. Mitchell WaldropMay 2002 Hook enough computers together and what do you get? A new kind of utility that offers supercomputer processing on tap. Is Internet history about to repeat itself?

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    Industrial Perspective on Grids:A Wide Range of ApplicationsSources: IDC, 2000 and Bear Stearns- Internet 3.0 - 5/01 Analysis by SAIGrid Services Market Opportunity 2005Unique by Industry with Common Characteristics

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    OverviewGrid: why and whatEvolution of Grid technologyOpen Grid Services ArchitectureFuture directionsTowards lightweight VOs: dynamic trust relationshipsTowards global knowledge communities: virtual data and dynamic workspaces

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    Open Grid Services ArchitectureService-oriented architectureKey to virtualization, discovery, composition, local-remote transparency Leverage industry standardsInternet, Web servicesDistributed service managementA component model for Web servicesA framework for the definition of composable, interoperable servicesThe Physiology of the Grid: An Open Grid Services Architecture for Distributed Systems Integration, Foster, Kesselman, Nick, Tuecke, 2002

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    Web Services XML-based distributed computing technologyWeb service = a server process that exposes typed ports to the networkDescribed by the Web Services Description Language, an XML document that containsType of message(s) the service understands & types of responses & exceptions it returnsMethods bound together as port typesPort types bound to protocols as portsA WSDL document completely defines a service and how to access it

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    OGSA StructureA standard substrate: the Grid serviceStandard interfaces and behaviors that address key distributed system issuesA refactoring and extension of the Globus Toolkit protocol suite supports standard service specificationsResource management, databases, workflow, security, diagnostics, etc., etc.Target of current & planned GGF efforts and arbitrary application-specific services based on these & other definitions

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    Open Grid Services InfrastructureImplementationHosting environment/runtime(C, J2EE, .NET, )DataaccessLifetime management Explicit destruction Soft-state lifetime

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    Open Grid Services InfrastructureGWD-R (draft-ggf-ogsi- gridservice-23) Editors:Open Grid Services Infrastructure (OGSI) S. Tuecke, ANLhttp://www.ggf.org/ogsi-wg K. Czajkowski, USC/ISII. Foster, ANLJ. Frey, IBMS. Graham, IBMC. Kesselman, USC/ISID. Snelling, Fujitsu LabsP. Vanderbilt, NASAFebruary 17, 2003

    Open Grid Services Infrastructure (OGSI)

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    Example:Reliable File Transfer ServiceFileTransferInternalStateData transfer operations

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    Open Grid Service Architecture:Next StepsTechnical specificationsOpen Grid Services Infrastructure is completeSecurity, data access, Java binding, common resource models, etc., etc., in the pipelineImplementations and compliant productsHere: OGSA-based Globus Toolkit v3, Announced: IBM, Avaki, Platform, Sun, NEC, HP, Oracle, UD, Entropia, Insors, , Rich set of service defns & implementations

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    Globus Toolkit v3 (GT3)Open Source OGSA TechnologyImplements OGSI interfacesSupports primary GT2 interfacesHigh degree of backward compatibilityMultiple platforms & hosting environmentsJ2EE, Java, C, .NET, PythonNew servicesSLA negotiation, service registry, community authorization, data management, Rapidly growing adoption and contributions: Linux for the Grid

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    OverviewGrid: why and whatEvolution of Grid technologyOpen Grid Services ArchitectureFuture directionsTowards lightweight VOs: dynamic trust relationshipsTowards global knowledge communities: virtual data and dynamic workspaces

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    Future DirectionsGrids are about computers, certainlyOn-demand access to computing, etc.Challenging future issues here: e.g., scale

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    CMS Event Simulation ProductionProduction Run on the Integration TestbedSimulate 1.5 million full CMS events for physics studies: ~500 sec per event on 850 MHz processor2 months continuous running across 5 testbed sitesManaged by a single person at the US-CMS Tier 1

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    CMS Event Simulation ProductionProduction Run on the Integration TestbedSimulate 1.5 million full CMS events for physics studies: ~500 sec per event on 850 MHz processor2 months continuous running across 5 testbed sitesManaged by a single person at the US-CMS Tier 1

    1.5 Million Events Delivered to CMS Physicists!

    (nearly 30 CPU years)

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    Future DirectionsGrids are about computers, certainlyOn-demand access to computing, etc.Challenging future issues here: e.g., scaleBut they are ultimately about people, their activities, and their interactionsNew interaction modalities supported by on-demand formation of lightweight VOsNew technologies needed: e.g., trust, security, data and knowledge integrationConvergence of interest between Compute and Collaboration Grids?

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    Global Knowledge Communities

    *foster@mcs.anl.govARGONNE CHICAGO

    Example Issue: Trust and SecurityEffective VO operation depends critically on Trust: can I rely on you?Protection mechanisms to govern actionsSuffers from VO-organization policy mismatchGoal: collaborations no longer defined by slow centralized mechanisms but canform spontaneously;be managed in a distributed manner; and be protected by an infrastructure that maintains and enforces trust relationships

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    Grid Security ServicesRequestorApplicationVODomainCredentialValidationServiceAuthorizationServiceRequestor'sDomainService Provider'sDomainAudit/Secure-LoggingServiceAttributeServiceTrustServiceServiceProviderApplicationBridge/TranslationServicePrivacyServiceCredentialValidationServiceAuthorizationServiceAudit/Secure-LoggingServiceAttributeServiceTrustServicePrivacyServiceCredentialValidationServiceAuthorizationServiceAttributeServiceTrustServiceCredentialValidationServiceAuthorizationServiceAttributeServiceTrustServiceWS-StubWS-StubSecure Conversation

    Grid, Globus Toolkit, and OGSA

    *foster@mcs.anl.govARGONNE CHICAGO

    Understanding and EnhancingVO Trust and Security trust ( Tr, Te, As, L )

Recommended

View more >