e-infrastructure shared between europe and latin america ist-2006-026409 - campina grande,...

E-infrastructure shared between Europe and Latin America

www.eu-eela.org - Campina Grande, OurGrid/EELA interoperability meeting 31.08.2006IST-2006-026409

Alexandre Duarte

OurGrid/EELA Interoperability Meeting

Campina Grande, 31.08.2006

MiddlewareThe

www.eu-eela.org - Campina Grande, OurGrid/EELA interoperability meeting 31.08.2006 2IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaObjective

Provide leverage on gLite to support a discussion about the interoperability between EELA (gLite) and OurGrid


E-infrastructure shared between Europe and Latin AmericaWhy ?

• Visibility– EELA gains visibility with OurGrid partners– OurGrid gains visibility with EELA partners

• Application/Users sharing– EELA needs more applications/users– OurGrid too

• Interesting thecnical problem• Good international cooperation opportunity

– EELA 2 (2008/2009)


E-infrastructure shared between Europe and Latin AmericaWhat is gLite?

• Lightweight Middleware for Grid Computing• Born from the collaborative efforts of more than 80 people

in 12 different academic and industrial research centers as part of the EGEE Project

The European Organization for Nuclear Research

(CERN) Istituto Nazionale di Fisica Nucleare

(INFN), Italy

Datamat Spa, Italy

CESNET, Czech Republic

The National Institute for Nuclear Physics

and High Energy Physics, The Netherlands

Universiteit van Amsterdam (UvA),

The Netherlands

Centre National de la Recherche Scientifique

(CNRS), France

CS Systeme d'Information (CSSI), France

Royal Institute of Technology, Center for Parallel

Computers (KTH-PDC), Sweden

University of Helsinki (UH.HIP), Finland

University of Bergen (UiB), Norway

Council for the Central Laboratory of the Research

Councils (CCLRC), United Kingdom


E-infrastructure shared between Europe and Latin AmericaCurrent Status

• Scale (June 2006):– ~ 200 sites in 40 countries– ~ 25 000 CPUs– > 10 PB storage– > 35 000 jobs per day– > 60 Virtual Organizations


E-infrastructure shared between Europe and Latin AmericaOverview

• The gLite Grid services follow a Service Oriented Architecture– Lightweigh services– Allow multiple interoperable implementations.– Easily and quickly deployable

• Use existing services where possible.– Condor, EDG, Globus, AliEn, etc

• Co-existence with other deployed infrastructure– eg. Interoperability with OSG, NorduGrid and OurGrid .

Globus 2 based Web services based

gLite-2gLite-1LCG-2LCG-1


E-infrastructure shared between Europe and Latin AmericagLite Job Submission

Computing Element

Storage Element

Site X

Information System

submit

submit

query

retrieve

retrieve

Workload Manager

User Interface

publishstate

File and ReplicaCatalogs

AuthorizationService

query

updatecredential publish

state

discoverservices



Job Description


E-infrastructure shared between Europe and Latin AmericaJob Description Language

• In gLite Job Description Language (JDL)Job Description Language (JDL) is used to describe jobs for execution on Grid.

• The JDL adopted within the gLite middleware is based upon Condor’s CLASSified Advertisement language CLASSified Advertisement language (ClassAd)(ClassAd).

• A ClassAd is a record-like structure composed of a finite number of attributes separated by semi-colon (;)

• A ClassAd is highly flexible and can be used to represent arbitrary services

www.eu-eela.org - Campina Grande, OurGrid/EELA interoperability meeting 31.08.2006 10

IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaJDL Example

Type = "Job";

JobType = "Normal";

Executable = "startGen4.sh";

Environment = {"CLASSPATH=./gfal.jar:./gint.jar","LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH","LCG_GFAL_VO=gilda","LCG_RFIO_TYPE=dpm"};

Arguments = " 0 0 10 4 10000 aliserv6.ct.infn.it lfn:/grid/gilda/valeria/2000pillar.dat /gilda/ischia06/vardizzo";

StdOutput = "sample.out";

StdError = "sample.err";

InputSandbox = {"startGen4.sh","gint.jar","gfal.jar","libGFalFile.so"};

OutputSandbox = {"sample.err","sample.out"};

Requirements = Member("GLITE-3_0_0",other.GlueHostApplicationSoftwareRunTimeEnvironment);


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaDAG JDL

• A DAG job is a set of jobs where input, output, or execution of one or more jobs can depend on other jobs

• Dependencies are represented through Directed Acyclic Graphs, where the nodes are jobs, and the edges identify the dependencies

nodeA

nodeB nodeC NodeF

nodeD


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaDAG JDL Example

[ type = "dag"; max_nodes_running = 4; nodes = [ nodeA = [ file ="nodes/nodeA.jdl" ; ]; nodeB = [ file ="nodes/nodeB.jdl" ; ]; nodeC = [ file ="nodes/nodeC.jdl" ; ]; nodeD = [ file ="nodes/nodeD.jdl"; ]; dependencies = { {nodeA, nodeB}, {nodeA, nodeC}, { {nodeB,nodeC}, nodeD } } ];]

Node description could also be done here,

instead of using separate files


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaJob Collection

• A job collection is a set of independent jobs that a user wants to submit and monitor via a single request

• Jobs of a collection are submitted as DAG nodes without dependencies

• JDL is a list of classad, which describes the subjobs

[

Type = "collection";

VirtualOrganisation = “gilda";

nodes = {

[ <job descr 1 >],

[ <job descr 2 >],

…

};

]


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaJob Collection Example

[ type = "collection"; InputSandbox = {"date.sh"}; RetryCount = 0; nodes = { [ file ="jobs/job1.jdl" ; ], [ [

Executable = "/bin/sh"; Arguments = "date.sh"; Stdoutput = "date.out"; StdError = "date.err"; OutputSandbox ={"date.out", "date.err"};]

], [ file ="jobs/job3.jdl" ; ] };]

All nodes will share this Input Sandbox


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaInput Sandbox

• Input Sandbox can contain – file paths on the UI machine (i.e. the usual way)– URI pointing to files on a remote gridFTP/HTTPS server

• A base URI to be applied to all sandbox files can also be specified

• Only local files (file://) are uploaded to the WMS node• File pointed by URIs are directly downloaded on the WN by the

JobWrapper just before the job is started

InputSandbox = {

"gsiftp://neo.datamat.it:2811/var/prg/sim.exe",

"https://ghemon.cnaf.infn.it:8443/data/idat_1",

"file:///home/pacio/myconf“ };

InputSandboxBaseURI = "gsiftp://matrix.datamat.it:2811/var";


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaOutput Sandbox

• JDL has been enriched with new attributes for specifying the destinations for the files listed in the OutputSandbox attribute list

• A base URI to be applied to all sandbox files can also be specified

• Files are copied when the job has completed execution by the JobWrapper to the specified destination without transiting on the WMS node

OutputSandbox = { "jobOutput",

"run1/event1",

"jobError" };

OutputSandboxDestURI = {

"gsiftp://matrix.datamat.it/var/jobOutput",

"https://grid003.ct.infn.it:8443/home/cms/event1",

"gsiftp://matrix.datamat.it/var/jobError" };

OutputSandboxBaseDestURI = "gsiftp://neo.datamat.it/home/run1/";


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericagLite Services

Access

CLI API

Security Services

Authorization

Authentication

Auditing

Information and Monitoring Services

Information & Monitoring

Job

Monitoring

Data Services

Metadata Catalog

File & Replica Catalog

Storage Element

Data Management

Job Management Services

Job

ProvenancePackage Manager

Computing Element

Workload Management

Accouting



Security ServicesSecurity Services

Authorization

Authentication

Auditing


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaAuthentication

• Identification of users and services– based on X.509 PKI infrastructure– Certificate Authorities (CA) issue (long lived) certificates

identifying individuals (much like a passport)– Trust between CAs and sites is established (offline)– In order to reduce vulnerability, on the Grid user identification is

done by using (short lived) proxies of their certificates

• Proxies can– Be delegated to a service such that it can act on the user’s

behalf– Include additional attributes (like VO information via the VO

Membership Service VOMS)– Be stored in an external proxy store (MyProxy) – Be renewed (in case they are about to expire)


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaAuthorization

• Allows or denies access to services, based on policies.– Agent: The user interacts with a centralized Authorization Source– Push: Authorization Services issue Tokens– Pull: The resource asks to the Authorization Services

• VOMS provides a way to add attributes to a certificate proxy:– produces a signed Attribute Certificate (AC)– The attributes are used to provide the user with additional

capabilities according to the VO policies


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaAuditing

• Monitoring and Post-Mortem analysis of security related events.

• In computational grids It goes hand by hand with the accounting– Who did what?– Where and when?

• In case of accounting:– For how long?– For how much?





Information & Monitoring

Job

Monitoring


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaInformation Services

• What is?– System to collect information on the state of resources

• Why?– To discover resources of the grid and their nature– To have useful data in order to who is in charge of managing the

workload to do it more efficiently.– To check for health status of resources.

• How?– Monitoring state of resources locally and publishing fresh data

on the information system.– Adopting a data model that MUST be well known to all

components that want to access monitored information


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaAdopted IS

• The BDII (Berkeley DB Information Index)– has been adopted in LCG middleware as the Information System

provider. – It is an evolution of the Globus Meta Directory System (MDS)– It is based on Lightweight Directory Access Protocol (LDAP)

servers.

• The Relational Grid Monitoring Architecture (R-GMA)– It is an implementation of the Grid Monitoring Architecture (GMA)

standardized by the Global Grid Forum (GGF, now OGF)– It is a relational implementation of the GMA– It is Web Services Oriented– It uses standard SQL query syntax


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaGRISs, local BDII and BDII

Each site can run

a BDII. It collects the information

given by the local BDIIs

At each site, a *local* BDII collects the information

given by the GRISs

Local GRISes run on CEs and SEs at each site and report dynamic and static information

Abbreviations:

BDII: Berkeley DataBase Information Index

GIIS: Grid Index Information

Server

GRIS: Grid Resource

Information Server





Job


Computing Element

Workload Management

Workload Management


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaWorkload Management

• The The Workload Management System (WMS) comprises a set of Grid middleware components responsible for distribution and management of tasks across Grid resources.

• The purpose of the Workload Manager (WM) is accept and satisfy requests for job management coming from its clients– meaning of the submission request is to pass the responsibility

of the job to the WM. WM will pass the job to an appropriate CE for execution

• taking into account requirements and the preferences expressed in the job description file

• The decision of which resource should be used is the outcome of a matchmaking process.


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaScheduling Policies

• Different possible policies – Eager scheduling: a job is bound to a resource as soon as

possible Job is then forwarded to that CE, where very likely it will end up in a

queue

– Lazy scheduling: job held by the WM until a resource becomes available Job then forwarded to that CE for immediate execution

• WM architecture able to accommodate both models (and the intermediate solutions)– Eager scheduling: matching a job against multiple resources– Lazy scheduling: matching a resource against multiple jobs


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaArchitecture

Job managementJob managementrequests (submission, requests (submission, cancellation) expressedcancellation) expressed

via a Job Descriptionvia a Job DescriptionLanguage (JDL)Language (JDL)

Finds an appropriateFinds an appropriateCE for each submission CE for each submission

request, taking into account request, taking into account job requests and preferences, job requests and preferences, Grid status, utilization policies Grid status, utilization policies

on resources on resources

Keeps submission Keeps submission requestsrequests

Requests are keptRequests are kept for a whilefor a while

if no resources are if no resources are immediately availableimmediately available

Repository of resourceRepository of resource informationinformation

available to matchmakeravailable to matchmaker

Updated via notifications Updated via notifications and/or active and/or active

polling on resourcespolling on resources

Performs the actual Performs the actual job submission job submission and monitoring and monitoring


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaJob Submission Services

• WMS components handling the job during its lifetime and performs the submission

• Job Adapter (JA)– is responsible for

making the final touches to the JDL expression for a job, before it is passed to CondorC for the actual submission

creating the job wrapper script that creates the appropriate execution environment in the CE worker node

• transfer of the input and of the output sandboxes

• CondorC– responsible for performing the actual job management

operations job submission, job removal


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaDAGMan

• Meta-scheduler • Purpose is to navigate the graph• Determine which nodes are free of

dependencies• Follow the execution of the corresponding jobs





Job


Computing Element

Workload Management

Computing Element


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaComputing Element

• Service representing a computing resource• Main functionality: job management

– Run jobs– Cancel jobs– Suspend and resume jobs– Provide info on “quality of service”

How many resources match the job requirements ? What is the estimated time to have the job starting its execution ?

• Used by the WM or by any other client (e.g. end-user)• CE architecture accommodated to support both push and pull model

– Push model: the job is pushed to the CE by the WM– Pull model: the CE asks the WM for jobs

• These two models are somewhat mirrored in the resource information flow– In order to 'pull' a job a resource must choose where to 'push'

information about itself


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaCE Architecture

CECECECE

LSFLSFLSFLSF

Worker NodesWorker NodesWorker NodesWorker Nodes

PBSPBSPBSPBS ????

MonMonMonMon

ClientClient

WEB WEB

JobSubmitJobAssess

JobKillJobSuspendJobResume

JobGetStatus

Web serviceaccepting jobmanagement

requests

Async. notificationsabout job/CE events

Job requests (forCE working in pullmode)


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaCE Implementations

• LCG-CE: based on GT2 GRAM – To be replaced when other CEs prove reliability

• gLite-CE: based on GSI enabled Condor-C– Supported by Condor. More efficient. – Deployed for the first time in gLite 3.0

• CREAM: new lightweight web service CE– Not in gLite 3 release. Will need exposure to users on dedicated

system.– WSDL interface– Will support bulk submission of jobs from WMS and optimization

of input/output file transfer.– Plan is to have a CE with both Condor-C and CREAM interfaces





Job


Computing Element

Workload Management

Job Information


IST-2006-026409


Logging and Bookkeeping

• Collects and manages job-related events (e.g. submission, suitable CE found, start of execution, …) from the WMS components

• Processes these events to give a higher level view on job states

• Both job states and raw data available to users– Also via Web Service interface

• Possible to subscribe to receive notifications on particular job state changes

• LB event trail can be analyzed to identify problems with resources ("black holes", unusual failure rates, etc).


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaJob Provenance

• Keeps track of definition of submitted jobs, execution conditions and job life cycle for a long time– Job life logs (JDL, timestamps, jobids, …)– Executable and input/output files– Execution environment (OS, installed software version, …)– Custom data provided by user

• Used for– Debugging– Post-mortem analysis– Comparison of job executions in an evolving environment

• Service components– Primary Storage Server

Keeps data in the most compact and economic form

– Index Servers Configured to support a set of query able attributes



Data ServicesData Services

Metadata Catalog


Storage Element

Data Management


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaData Management Services

• Data Management System is the subsystem of the gLite Middleware which takes care about data manipulation for both all other GRID services and user application.

• DMS provides all operation that users can perform on the data.– Creating files/directories– Renaming files/directories– Deleting files/directories– Moving files/directories– Listing directories– Creating symbolic links– Etc …..


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaSystem Architecture

• Data Management System is composed by three main modules: Storage Element, Catalog and File Transfer Service.

• The Storage Element takes care about data manipulation in order to make user and/or application to manage its own files.– Storage Resource Manager(SRM) : Castor, dCache, DPM, …– POSIX-I/O: GFAL – Native Access protocols: rfio, dcap, …– Transfer protocols: gsiftp, ftp, …


IST-2006-026409


• The File & Replica Catalog has in charge to keep trace about file location into the distributed file system and store any kind of information about files.– File Catalog– Replica Catalog– File Authorization Service– Metadata Catalog

• The File Transfer Service enables the GRID to move file from/to a site. It is a kind of intra-GRID moving file service.– Data Scheduler– File Transfer Service

manages physical transfers

– File Placement Service FTS and catalog interaction in a transactional way


IST-2006-026409




Data Services

Data Services

Metadata Catalog


Storage Element

Data Management

Storage Element


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaStorage Element

• SE has three external interfaces:– SRM– A native POSIX-like file I/O interface– GridFTP


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaStorage Resource Management

• SRM is a Grid Service that takes care of local storage interaction and provides a Grid interface to outside world– Data are stored on disk pool servers or Mass Storage Systems

• Storage resource management needs to take into account– Transparent access to files (migration to/from disk pool)– Space reservation– File status notification



Data Services

Data Services

Metadata Catalog


Storage Element

Data Management



IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaFile & Replica Catalog

• Users and applications need to locate files (or replicas) on the whole Grid. The File Catalog is the service which allows it and maintains the mappings between LFNs, GUIDs and SURLs.

• In gLite, file cataloguing operations are provided by the LFC (LCG File Catalog);

• LFC is deployed as a centralized service and its endpoint is published on the Information Service in order to be found by the DMS tools and/or other GRID services.


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaFile names and identifiers

Globally unique

identifier

Site URL

Transport URL: includes protocol

user need only see

these


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaFor More Information

http://www.glite.orghttp://www.eu-egee.org/http://www.eu-eela.org/http://www.cern.ch


IST-2006-026409

E-infrastructure shared between Europe and Latin AmericaCredits

• This presentation is based on previous work from:– Claudio Grandi– Roberto Barbera– Grupo Grid, ULA– Riccardo Bruno– Pedro Bello– Diego Scardaci

e-infrastructure shared between europe and latin america ist-2006-026409 - campina grande,...

Documents