4b.1 grid computing software components of globus 4.0 itcs 4010 grid computing, 2005, unc-charlotte,...

4b.1

Grid Computing Software

Components of Globus 4.0

ITCS 4010 Grid Computing, 2005, UNC-Charlotte, B. Wilkinson, slides 4b

4b.2

Globus Version 4

• A “toolkit” of services and packages for creating the basic grid computing infrastructure

• Higher level tools added to this infrastructure• Version 4 is web-services based• Some non-web services code exists from

earlier versions (legacy) or where not appropriate (for efficiency, etc.).

4b.4

• Each part comprises a set of web services and/or non-web service components.

• Some built upon earlier versions of Globus.

Data Management

SecurityCommonRuntime

Execution Management

Information Services

Web Services

Components

Non-WS

Components

Pre-WSAuthenticationAuthorization

GridFTP

GridResource

Allocation Mgmt(Pre-WS GRAM)

Monitoring& Discovery

System(MDS2)

C CommonLibraries

GT2

WSAuthenticationAuthorization

ReliableFile

Transfer

OGSA-DAI[Tech Preview]

GridResource

Allocation Mgmt(WS GRAM)


System(MDS4)

Java WS Core

CommunityAuthorization

ServiceGT3

ReplicaLocationService

XIO

GT3

CredentialManagement

GT4

Python WS Core[contribution]

C WS Core

CommunitySchedulerFramework

[contribution]

DelegationService

GT4

Globus Open Source Grid Software

I Foster

4b.6

Java Services in Apache AxisPlus GT Libraries and Handlers

YourJava

Service

YourPythonService

YourJava

Service RF

T

GR

AM

Del

egat

ion

Inde

x

Trig

ger

Arc

hive

r

pyGlobusWS Core

YourC

Service

C WS Core

RLS

Pre

-WS

MD

S

CA

S

Pre

-WS

GR

AM

Sim

pleC

A

MyP

roxy

OG

SA

-DA

I

GT

CP

Grid

FT

P

C Services using GT Libraries and Handlers

SERVER

CLIENT

InteroperableWS-I-compliant

SOAP messaging

YourJavaClient

YourC

Client

YourPythonClient

YourJavaClient

YourC

Client

YourPythonClient

YourJavaClient

YourC

Client

YourPythonClient

YourJavaClient

YourC

Client

YourPythonClient

X.509 credentials =common authentication

Python hosting, GT Libraries

Another view of GT4 Components

I Foster

4b.7

GT Core

• Provides the ability to create services running inside the GT 4 container.

• Assignment 2 requires you to create a service inside GT 4 container and exercise it with a client.

Data Management




Web Services

Components

Non-WS

Components


GridFTP

GridResource



System(MDS2)

C CommonLibraries

GT2


ReliableFile

Transfer


GridResource



System(MDS4)

Java WS Core


ServiceGT3


XIO

GT3


GT4


C WS Core


[contribution]

DelegationService

GT4

Java WS Core

Used in assignment 2

4b.9

CustomWeb

ServicesWS-Addressing, WSRF,

WS-Notification

CustomWSRF Web

Services

GT4WSRF Web

Services

WSDL, SOAP, WS-Security

User Applications

Reg

istr

yA

dmin

istr

atio

n

GT

4 C

onta

iner

GT4 Web Services Core

I Foster

4b.10


Key component

GRAM (Grid Resource Allocation Manager)

• For submitting executable jobs• Used in Assignment 3 to submit and execute

jobs.• May interface to a local job scheduler• Local job scheduler used in assignment 4

Data Management




Web Services

Components

Non-WS

Components


GridFTP

GridResource



System(MDS2)

C CommonLibraries

GT2


ReliableFile

Transfer


GridResource



System(MDS4)

Java WS Core


ServiceGT3


XIO

GT3


GT4


C WS Core


[contribution]

DelegationService

GT4

GRAM (Grid Resource Allocation Manager)

Used in assignment 3

4b.12

GRAMservices

GT4 Java Container

GRAMservices

Delegation

RFT FileTransfer

Transferrequest

GridFTPRemote storage element(s)

Localscheduler

Userjob

Compute element

GridFTP

sudo

GRAMadapter

FTPcontrol

Local jobcontrol

Delegate

FTP data

Cli

ent

Job

functions

Delegate

Service host(s) and compute element(s)

GT4 GRAM Structure:

Sun Grid Engine used in assignment 4

Data management components

I Foster

4b.13

Security ComponentsAddresses the security requirements of grid computing. Three important factors are:

• Authorization– Process of deciding whether a particular identity can

access a particular resource

• Authentication– Process of deciding whether a particular identity is

who he says he is (applies to humans and systems)

• Delegation (somewhat specific to grid computing)– Process of giving authority to another identity

(usually a computer/process) to act on your behalf.

4b.14

Security continued

• Security aspects complicated by the fact that virtual organization members and resources can be in different administrative domains.

Data Management




Web Services

Components

Non-WS

Components


GridFTP

GridResource



System(MDS2)

C CommonLibraries

GT2


ReliableFile

Transfer


GridResource



System(MDS4)

Java WS Core


ServiceGT3


XIO

GT3


GT4


C WS Core


[contribution]

DelegationService

GT4

Security

4b.17

GT4’s Use of Security Standards

I Foster

4b.18

GT4 Data Management

• Move large data to/from nodes• Replicate data for performance &

reliability• Locate data of interest• Provide access to different data sources

– File systems, parallel file systems, hierarchical storage (GridFTP)

– Databases (OGSA DAI)

Data Management




Web Services

Components

Non-WS

Components


GridFTP

GridResource



System(MDS2)

C CommonLibraries

GT2


ReliableFile

Transfer


GridResource



System(MDS4)

Java WS Core


ServiceGT3


XIO

GT3


GT4


C WS Core


[contribution]

DelegationService

GT4

GridFTP and Reliable File Transfer

4b.20

GridFTP• Built on FTP using separation of data and

control channels• Provides features for

– Large data transfers– Secure transfers– Fast transfers– Reliable transfers– Third party transfers

• Not a web service– RTF (Reliable File Transfer) service provided WS-

level interface

4b.21

Third party transfers

PI = FTP Protocol InterpreterDTP= FTP Data Channel Process

PI

DTP DTP

PI

PI PI

Client

Server Server

Control channels

Data channel

4b.22

Performing a third-party transfer

1. Client establishes control channel with server2. Using control channel, client sets up transfer

parameters and requests data channel creation

3. Data channel established,4. Client sends transfer command over control

channel,5. Data transfer starts through data channel.

Either client or server can send.

4b.23

Parallel transfers and striping

• Using multiple (virtual) connections for transfer– Same external network– Speed improvement possible, but limited by

network card

• Striping– a version of parallel transfers that can use

separate hardware interfaces– Implemented in GT 4.

4b.24

GridFTP and RFT

WS ClientRFT service

(Java)

Client API(Java)

XIO based (C) XIO based (C)

Control channel

Data channel

Control channel

GridFTP server GridFTP server

From Gridwise

4b.25

GT 4 Replica Location Service

• Identify location of files via logical to physical name map

• Distributed indexing of names, fault tolerant update protocols

IndexIndex

I Foster

Data Management




Web Services

Components

Non-WS

Components


GridFTP

GridResource



System(MDS2)

C CommonLibraries

GT2


ReliableFile

Transfer


GridResource



System(MDS4)

Java WS Core


ServiceGT3


XIO

GT3


GT4


C WS Core


[contribution]

DelegationService

GT4

Monitoring and Discovery

4b.27

Monitoring and Discovery• WSRF provides common mechanisms for

monitoring and discovering a service:• GT4 “aggregator” services within MDS:

– MDS-Index: collects state information from registered resources and makes it available as XML document

– MDS-Trigger: passes this information to an executable

– MDS-Archive: archives state information (awaiting implementation)

• Every GT 4 is discoverable

4b.28

AcknowledgementSlides numbers marked with “I Foster” have been

selected from presentations made by Ian Foster:• Enabling eScience: Grid Technologies Today &

TomorrowAmerican Association for the Advancement of Science Annual Meeting, Washington, DC, February 21 2005.

• Globus: Bridging the GapKeynote Talk, GlobusWORLD, Boston, Mass., February 8, 2005.

• The Grid: Reality, Technologies, ApplicationsDistinguished Lecture, McGill University, Montreal, Canada, January 21 2005.

used for educational purposes only.

4b.1 grid computing software components of globus 4.0 itcs 4010 grid computing, 2005, unc-charlotte,...

Documents