Grid Middleware, Grid Canada

darcy.quesnel@canarie.ca

Globus Toolkit 4

Background

> Open source project> Under development since

1996> Hosted by

– DOE Argonne National Laboratory– USC Information Sciences

Institute

> Now overseen by the Globus Alliance

– Argonne, ISI– EPCC, University of Edinburgh– NCSA– KTH, Sweden– Univa Corporation

> Funded by– NSF– IBM– DOE– NASA– DARPA– European Commission– UK Research Councils– Microsoft

Background

> Commercial partners in the Globus Consortium– HP– IBM– Intel– Sun– Nortel– Univa

> Enterprise Grid Alliance– Data Synapse– EMC2

– Fujitsu– HP– Intel– NEC– NetApp– Oracle– Sun– …

Background

> Various versions of Globus in use by a number of “large science” projects– Sloan Digital Sky Survey– National Virtual Observatory– Collaboratory for Multi-Scale Chemical Science– Network for Earthquake Engineering Simulation– Earth System Grid– Access Grid– Science Environment for Ecological Knowledge– Laboratory for the Ocean Observatory Knowledge Integration Grid– Particle Physics Data Grid– NAREGI– Enabling Grids for e-Science (EGEE)– Open Science Grid– TeraGrid– UK e-Science– …

Components

GT4 Security

RightsUsers

Rights

ComputeCenter

Access

Services (runningon user’s behalf)

Rights

Local Policyon VO identityor attributeauthority

CAS or VOMSissuing SAMLor X.509 ACs

SSL/WS-Securitywith ProxyCertificates

Authz Callout

KCA

MyProxyVO

GT4 Security

Data Management

> Discover, transfer, and access large data sets> Stage large data sets to/from nodes and provide

access to diverse data sources, file systems, parallel file systems, and hierarchical storage, GridFTP

> Replicate data for performance and reliability, RFT> Locate data of interest, RLS> Provide access to databases, OGSA-DAI

Execution Management

> Provision, deploy, and manage services> Common WS interface to schedulers

– Unix, Condor, LSF, PBS, SGE, …

> More generally: interface for process execution management– Lay down execution environment – Stage data– Monitor and manage lifecycle– Kill it and clean up

> A basis for application-driven provisioning> This is where WSRF and WS-N come in

Information Services

> Discover and monitor dynamic services> “Every service should be monitorable and discoverable using

common mechanisms”– WSRF/WSN provides those mechanisms

> A common aggregator framework for collecting information from services, thus:– MDS-Index: XPath queries, with caching– MDS-Trigger: perform action on condition– MDS-Archiver: XPath on historical data

> Deep integration with Globus containers & services: every GT4 service is discoverable– GRAM, RFT, GridFTP, CAS, …

GT4 In Action

IBM

IBM

GRAM GridFTPHost Env

User Svc

ReliableFile

Transfer

MyProxy

Uniform interfaces,security mechanisms,Web service transport,

monitoring

Host Env

User Svc

Computers Storage

UserApplication

UserApplication

UserApplication

IBM

IBM

DAIS

Database

MDS-Index

ToolTool

Implementations

> Multiple implementations of the web services parts (WSRF, WSN)– GT4-Java, Java– GT4-C, C– pyGridWare, Python– WSRF::Lite, Perl– WSRF.Net, C#

Ecosystem

> MPICH-G2, MPI in the wide-area> Nimrod-G, parameter studies> Ninf-G, remote procedure calls> Condor-G, reliable job submission> Grid Resource Management System, metascheduler> Open Grid Computing Environment, portal

Grid Canada

Grid Canada Drivers

ATLASLHC

LHC

SAFORAHCanadian Forestry Grid

Canadian Grid Groups

> Grid Research Centre at U Calgary

– Rob Simmonds, Brian Unger, &c

> Grid Group at U Windsor– Bob Kent, A.K. Aggarwal, &c

> Atlas Canada– Randy Sobie, U Victoria, Bryan

Carron, U Alberta, Michel Vetterli, TRIUMF, &c

> CRIM– Pierre Duchoumel, &c

> International Grid Testbed– Bryan Carron, U Alberta,

Wade Hong, Carleton U, &c

> NRC Internal HPC Programme / Multi-Scale Modelling Project– Roger Impey, &c

> SPARRO at U Regina– Edward Brash, &c

> UQAM– Omar Cherkaoui, &c

Grid Canada

> Informational and consultative functions, a forum for interested parties– www.gridcanada.ca

> giis.gridcanada.ca– Information service, resources attached to the Grid Canada

testbed and a production grid effort, Grid-X1

> www.gridcanada.ca/ca/– Grid Canada Certificate Authority– myproxy.gridcanada.ca

> We’re looking at VO management

Grid CanadaCertificate Authority

> Issues certificates to users and resources involved in grid-related research in Canada– Atlas Canada, NRC Grid, WestGrid, and anyone else who wants

to talk to us (!)

> Certificates are used in the context of PKI as implemented by grid middleware such as Globus

> CA operations are (securely) housed at NRC with Ratilal Haria handling day-to-day certificate signing

> Process is documented in our CP/CPS– http://www.gridcanada.ca/ca/– Many legal issues side-stepped or “not stipulated”

GC CAIssued Certificates

> Around fifteen hundred issued certificates– Mainly Westgrid users (!)

> Not all are active– User and host certificates– Some are revoked– Many are expired

• Certificates expire after a year but can be renewed

> We’ve been issuing certificates since April 2002

GC CAExternal Relations

> Trust between CA’s about the processes which we follow> Certificates from one CA are “allowed” to authenticate with

those of another> GC CA has trust relationships with all the CA’s associated with

the EU Grid PMA– http://www.eugridpma.org/– In response to the need for Canadian researchers to access

international resources such as LCG

> GC CA is listed in the TERENA TACAR, a reliable repository

GC CA External Relations

> The Americas Grid Policy Management Authority (TAGPMA)– Grid Canada, DOEGrids, FermiLab, &c– Main drivers are large projects such as OSG and Teragrid– Canadian researchers will be able to access TeraGrid resources

soon with GC certificates

GC CA External Relations

> Global Grid Forum, Certificate Authority Operations Working Group– Documents, both finished and in the pipeline– Deal with PMA issues as well as a publishing point– www.ggf.org

GC CAExternal Relations

> International Grid Trust Federation> Formed at the last GGF in Boston> Regroups EUGridPMA, APGridPMA, and TAGPMA> Allows for international trust to be established> This represents 10’s of thousands of users, and 10’s

of thousands of resources

Questions?

> Contact me– darcy.quesnel@canarie.ca