Managing Computational Activities on the Grid - from Specifications to Implementation:

The GLUE 2 information model

OGF25, 2nd March 2009, Catania

Balázs Kónya, Lund University/NorduGrid

Special thanks to Sergio Andreozzi, co-chair of the OGF GLUE WG, for many of the slides

Agenda• Context and Problem Description

• Pre-Glue 2 schemas – GLUE 1.X (2002-2006)– NorduGrid schema (2001-)

• Glue2 Working Group of Open Grid Forum– Birth of a specification

• Insight on GLUE 2 Info model

Context and Problem Description

Grid as a multi-institutional infrastructure

Intra-site resource

local credential

Inter-site seamless collaboration

Grid-level credential

Virtual organization

Need for a com

on language!

Problem Statement

• How do we describe resources shared in Grid systems in order to enable:

– Resource awareness

– Resource discoverability

– Resource requirements expression

– Resource basic monitoring

Use Case 1 (discovery)• I want to run my job on an

execution environment characterized by:

– OS• Linux, Distribution X, version Y

– CPU Archicture • IA64

– Available software packages: • S1, S2

Use Case 2 (monitoring)

• I want to know

– how many job slots are used by members of the VO A

– what is the global available storage space for the users of VO B

Grid Information SystemWhere can I run a job

requiring OS Linux, IA64 architecture, with software

package X and Y

As part of the VO A, how much storage can I use on

the Grid?

I can offer IA64 machines with OS Linux using BES

interface to users of BLUE VO

I offer 15 TB of storage, 10 TB are free and usable by

GREEN VO

Generalization capture common aspects for different entities providing the same

functionality e.g.: uniform view over different batch services

Abstraction given by the Grid paradigmVirtual pool of resourcesGrid-related user attributes (e.g., VO, groups, roles)

Main focus on discovery for brokering, monitoring and inventoryconcerns those attributes that are meaningful for locate resources on the

basis of a set of preferences/constraints

Avoid publishing unnecessary local information

Modeling Guidelines

Pre-GLUE 2 Schemas

Situation Before GLUE 2

• Middleware vendors were forced to define their own information model• Globus: MDS schema• NorduGrid: NorduGrid schema• Glite: Glue 1.x schema• Condor: classadds• CIM

• Grid infrastructures deploying a middleware extended/mixed some of these schemas• E.g. Globus MDS could only describe a computing node

• To bridge the gap, translators were created• Even OGF specifications created there own embeded information model

• JSDL• BES

• For interoperable Grids, we need to unify the modeling of Grid resources into a community standard

GLUE Schema 1.x • Collaborative effort focusing on interoperability started by the EU DataTAG and US

iVDGL Grid projects

• Initial Contributors: DataGrid, Globus, PPDG, GriPhyn, NorduGrid

• Goal: – a common description for Grid resources designed to support discovery and selection via Grid

information Service

• Version 1.3 was released December 2006

• Still heavily used in production by EGEE and OSG Grid

01/04/2002 31/12/2006

01/01/2003 01/01/2004 01/01/2005 01/01/2006

9/2002GLUE 1.0

4/2003GLUE 1.1

5/2005GLUE 1.2

12/2006GLUE 1.3

GLUE 1.X - conceptsCore

Site, Service, ElementComputing

Cluster/SubCluster/HostComputing Element

StorageStorage ElementStorage AreaAccess/Control Protocol

NorduGrid Schema

• Used in production since May 2001• Formulated as an LDAP schema• Models computing elements by giving natural

representation of– Clusters– Queues– User-specific information– Grid jobs

• Basic model of Storage Element also exists

Nordugrid Schema objectscluster

queue

jobs users

job-01 job-02 job-03 user-01 user-02queue

jobs users

job-04 job-05 user-02 user-03user-01

GLUE 2 Working Group of OGF:birth of a specification

OGF GLUE WG• A new OGF Working Group was approved at OGF19 (Jan 2007)

• Previous Glue activity was moved under the OGF umbrella

• Co-chairs: – Sergio Andreozzi (OMII-Europe)– Laurence Field (EGEE)– Balazs Konya (NorduGrid)

• Focus: – facilitate interoperability between Grid infrastructures via common information

models and reference implementation for describing Grid resources in response to use cases

• Goal: – define a use case document collecting use cases from different Grid

projects/infrastructures– define a conceptual model defining the abstract schema GLUE 2.0 satisfying the

collected use cases. – develop reference implementations

• Unify modeling approaches and experience in production systems• Bring information modeling to a common platform

Contributors & Adapters

The definition of the GLUE 2 Info Model was an open process:End-users (persons using Grid systems)Site administratorsGrid operatorsVirtual Organizations managersMiddleware Developers

Early adopters:Glite, ARC, Unicore, TeraGrid, ...See Glue 2 Implementation session on Wednesday

Timeline: planning vs. reality

01/01/2007 30/04/2008

01/01/2008

1/2007OGF19 - 1st GLUE WG Session

5/2007OGF20 - Entity and Use Cases Session

10/2007OGF21 - Mature Spec and Use Cases Docs

11/2007SC2007 - BES + GLUE interop demo

1/2008OGF22 – Docs in Public Comment

& Ref. Implem. available

• June 2008: GLUE 2 specification entered the public comment period• August 2008: public comment period ended• February 2009 (OGF25): GLUE 2 final version submited to OGF Editor• April 2009: Rendering documents to be submitted to the OGF Editor

GLUE 2 documents

• GLUE Specification – v.2.0• Conceptual model in three sub-models

– Main Entities– Computing Entities– Storage Entities

• Final version submitted to OGF Editor • GLUE v. 2.0 – Reference Realizations to Concrete Data Models

• XSD• SQL• LDAP

• Public comment version is available, not yet updated to the final version• GLUE Use Cases – live document

• Available on the gridforge

Insight on the GLUE 2 Model

What is inside GLUE 2?• Entities

– Description– Attributes

• Type (GLUE types)• Multiplicity• Unit

• Relation of Entities– UML– Associations

• Misc.– Extension hooks– Defaults

Main Entities

• Entity• Extension• Location• Contact• Domain

– AdminDomain

– UserDomain

• Service• Endpoint• Share• Manager• Resource• Activity• Policy

– AccessPolicy

– Mappingpolicy

Computing entities

• ComputingService

• ComputingEndpoint

• ComputingShare

• ComputingManager

• Bencmark

• ExecutionEnvironment

• ApplicationEnvironment

• ApplicationHandle

• ComputingActivity

• ToStorageService

OpenPBS

#50 P4 2 GHz, 1 GB RAM #50 Xeon 5160 2.66 GHZ, 4 GB RAM

CREAM

BLUE VO GREEN VO

AdminDomain

AdminDomain

UserDomain

ComputingManager

ExecutionEnvironment

GLUE 2.0concepts

ComputingService

ApplicationEnvironment

ComputingEndpoint

ComputingShare

Complex Computing Service

CREAM-BES

blueshare

greenshare

Storage entities

• StorageService

• StorageServiceCapacity

• StorageAccessProtocol

• StorageEndpoint

• StorageShare

• StorageShareCapacity

• ToComputingService

• StorageManager

• DataStore

10 T B D isk

S toR M

B L UE VO G R E E N VO

greenshare

blueshare

G L UE 2.0concepts

AdminDomain

G L U E 2.0: S tora g e E ntities by E xa mple

G P F S

A d m in D o m a in

U s e rD o m a in

S to ra g e M a n a g e r

D a ta S to re

S to ra g e S e rv ic e

S to ra g e E n d p o in t

S tro a g e S h a re

Next stepsWe have an OGF-approved specification to represent grid entities

BUT:• Renderings are yet to be finalised• Implementations, production deployments will give lots of

feedback• Glue is an abstract model therefore

– No instructions on how to publish/obtain information– Howto consume information

• Profiles are needed to synchronize GLUE with other specifications• BES• JSDL

• Production Grid Infrastructure Profile (PGI)

References

• OGF GLUE Working Grouphttp://forge.gridforum.org/sf/sfmain/do/viewProject/projects.glue-wg

• GLUE 2.0 Documents• Specification:

http://forge.gridforum.org/sf/go/doc15023?nav=1

• Renderings:http://forge.gridforum.org/sf/go/doc15221?nav=1

• Use Cases: http://forge.gridforum.org/sf/go/doc15165?nav=1