The Anatomy of the Grid: An Integrated View of Grid

Architecture

Ian Foster, Steve TueckeArgonne National Laboratory

The University of Chicago

Carl KesselmanUSC/Information Sciences Institute

Outline and Goals

Define what is (and is not) a Grid in terms of a layered protocol architecture

Understand the role of protocols, services, APIs and tools in the deployment and use of Grids

Illustrate how a layered protocol architecture works with specific examples

Propose a view of where the Grid Forum can play a particularly important role

What is a Grid?

Grid language has been driven by genesis from metacomputing, but…

In practice, the Grid is about resource sharing and coordinated problem solving in dynamic, multi-institutional virtual organizations

Focus on how to enable, maintain and control the sharing of resources to achieve a common goal

Some Useful Definitions

Network Protocol A formal description of message formats and

a set of rules for exchange of messages Rules define sequences of message

exchange, and potentially resulting behavior Protocol may define state-change in endpoint

Network Enabled Services Defines a set of capabilities Protocol defines interaction with service

All services require protocols, although not all protocols are to services

More definitions

Resource Entity that is to be shared Provides some capabilities, that can be

accessed via interface (API) or protocol Application Programmer Interface (API) Software Development Kit (SDK)

Package that enables application development, consisting of one or more APIs, and programming tools

Protocols Make the Grid

Protocols vs. APIs Protocols enable interoperability APIs enable portability

Sharing is about interoperability, so … Grid architecture should be about protocols

By extension, a major role for the Grid Forum is to define protocols where standards do not exist

Grid Services Architecture: Previous Perspective

Grid-enabled archives, networks, computers, display devices, etc.; associated local services

Protocols, authentication, policy, resource management, instrumentation, discovery, etc., etc.

GridFabric

GridServices

ApplnToolkits

Applns

...

… a rich variety of applications ...

Remoteviz

toolkit

Remotecomp.toolkit

Remotedata

toolkit

Remotesensorstoolkit

Async.collab.toolkit

Characteristics of Grid Services Architecture

Identifies separation of concerns Isolates Grids from languages and specific

programming environments Makes provisions for generic and

application specific functionality Protocols not explicit in architecture

Fails to make clear distinction between language, service, and networking issues

Grid Services Architecture v2.0:Layered Grid Protocol Architecture

Connectivity

Fabric

Resource

Grid

Application

User

Important Points

Being Grid-enabled requires speaking appropriate protocols Protocol only requirement, not reachability Protocols can be used to bridge local resources

or “local Grids” Talk about Intergrid (analog to Internet)

Built on Internet protocols Independent of language and implementation

Focus on interaction over network Services exist at each level

Grid Services Architecture v2.0:Protocols, Services, and Interfaces

Languages/Frameworks

Fabric Layer

Applications

Local Access APIs and protocols

Grid Service APIs and SDKs

Grid ServicesGrid Service Protocols

Resource APIs and SDKs

Resource ServicesResource Service Protocols

User Service ProtocolsUser Service APIs and SDKs

User Services

Connectivity APIs

Connectivity Protocols

The Integrated Grid Architecturein Practice

We now proceed to illustrate the general concepts outlined above by reference to specific examples First, we analyze some extant systems in

terms of this architecture Then, we review example protocols,

services, APIs, SDKs in each layer Apology: We will necessarily need to refer

to some specific protocols and systems

Example: A User Portal

This to be filled in showing the different services protocols APIs etc. used at each layer

Example:Network Weather Service

ditto

Example: A Data Grid

ditto

Layer 1: Fabric

Local access to logical resource May be real component, e.g. CPU, software

module, filesystem May be logical component, e.g. Condor pool

Protocol or API mediated Fabric elements include

SSP, ASP, peer-to-peer, Entropia-like, and enterprise level solutions

Layer 2: Connectivity Protocols

Two classes of connectivity protocols underlie all other components

Internet communication Application, transport and internet layer

protocols I.e., transport, routing, DNS, etc.

Security Authentication and delegation Discussed below

Connectivity Layer Element:Grid Security Protocols

Protocols TLS with delegation

Services K5ssl, Globus Authorization Service

APIs GSS-API, GAA, SASL, gss_assist

SDKs GlobusIO

Layer 3: Resource Protocols

Resource management Storage system access Network quality of service Data transport Resource information

Resource Management

Protocols GRAM+GARA (on HTTP)

Resource services Gatekeeper, JobManager, SlotManager

APIs and SDKs GRAM API, Java CoG Kit Client, DUROC

Data Transport

Protocols Grid FTP, LDAP for replica catalog

Services FTP, LDAP replica catalog

APIs and SDKs GridFTP client library, copy URL API, replica

catalog access, replica selection

Resource Information

Protocol LDAP V3, Registration/Discovery protocol

Service GRIS

APIs & SDKs C API; JNDI, PerlLDAP, ….

Layer 4: Grid Protocols

Grid Information Index Services LDAP and Service registration protocol, … GIIS service LDAP APIs and specialized information API

Co-allocation and brokering GRAM (HTTP+RSL) DUROC service DUROC client API, end-to-end reservation

API

Grid Protocols (cont)

Online authentication, authorization services HTTP MyProxy, Group policy servers MyProxy API, GAA API,

Many others (e.g.): Resource discovery (Matchmaker) Fault recovery

Layer 5: User Protocols

In general, there are many of these, they tend to be one off and not standardized

Examples: Portal toolkits (e.g., Hotpage) Netsolve Cactus framework

Role of Standards

Standard APIs: code reuse, portability Certainly a good thing!

Standard protocols enable resource sharing via interoperability of infrastructure The fundamental problem

Note that Grid Forum is already doing this! Scheduling: remote access protocol Data: Grid FTP Security: Grid Security protocols Information: Standard info representations

Summary

Grids are about sharing, not making big MPI jobs

Well-defined protocol architecture is essential to understanding and progress Provides a framework for figuring out where

the pieces fit Grid Forum can play an important role by

helping to fill in missing pieces But note that protocol design is not for the

nonspecialist (or fainthearted)!