building components for grid interoperability

Building components for Grid InteroperabilityStephen Brewer, Deputy Project Manager, OMII-Europe

[email protected] 23, Barcelona, 2nd June 2008

2EU project: RIO31844-OMII-EUROPE

Outline

• What is OMII-Europe – Overview of the project– Vision and Objectives

• Approaches to Interoperability

• Key project results


What is OMII-Europe• OMII-Europe stands for

– Open Middleware Infrastructure Institute for Europe• It was an EU-funded project: FP6, RI• It had a duration of 2 years

• May 2006 -> April 2008• It was granted a contribution of 8M €• It involved 16 partners

– 8 EU– 4 USA– 4 China


Partners

Funded Unfunded

University of Southampton (coordinator) UK University of Chicago USA

Fujitsu Laboratories Europe UK NCSA, University of Illinois USA

Kungl Tekniska Högskolan Sweden University of Southern California,Los Angeles

USA

Istituto Nazionale di Fisica Nucleare Italy University of Wisconsin USA

Poznan Supercomputing and Networking Center

Poland Beihang University China

Forschungszentrum Juelich (FZJ) Germany China Institute of Computing Technology, Beijing

China

University of Edinburgh UK Computer Network Information Centre, Beijing

China

CERN, European Organisation for Nuclear Research

Switzerland Tsinghua University China


Project Structure and Effort Allocation

• Networking activities– Management, Outreach, Training– 8% Person Effort

• Service Activities– Repository, QA, Support– 25% Person Effort

• Joint Research Activities– Re-engineering, new services, integration,

benchmarking– 67% Person Effort


Vision

“ e-Science having easy access and use of Grid resources in heterogeneous e-infrastructures crossing national, pan-European and global boundaries “


Mission

“ Enabling of e-infrastructure interoperability by providing standards-based middleware components leveraging existing work and activities “


Focus

• Achieving interoperability through common standards– Common standards is the long term solution– Significant involvement and success in OGF and

Oasis– Implementations of standards in tandem with

standards development on all middleware platforms


Approaches to Interoperability

• Adapters-based:– The ability of Grid

middleware to interact via adapters that translate the specific design aspects from one domain to another

• Standard-based:– the native ability of Grid

middleware to interact directly via well-defined interfaces and common open standards

* definition inspired by OGF GIN CG


Who Benefits from Interoperability?

• Grid Developers– A single standard set of services on all Grid middleware systems– Applications portable across different Grid middleware systems

• E-Science application users– Common ways for accessing any e-infrastructure resources– Potential access to a significantly larger set of resources

• E-resource owners– Reduced management overheads as only a single Grid

middleware system needs deployment– Potential for greater resource utilisation

“For the Grid to deliver on it’s promises interoperability needs to be taken for granted like network interoperability”


Participation in Middleware Standardisation• Most project participants were involved as member/observer in many OGF WG• 11 project participant hold senior positions in

– OGSA DAIS WG (Database Access and Integration Services)– OGSA RUS WG (Resource Usage Server)– OGSA BES WG (Basic Execution Service)– OGSA JSDL WG (Job Submission Description Language)– GIN CG (Grid Interoperability Now)– OGSA-AuthZ-WG (Authorization)– GLUE WG – GFSG WG (Grid File System)– RM WG (Reference Model)– OGSA Naming WG– Technical Standards Committee– GSA RG (Grid Scheduling Architecture)– GRAAP WG (Grid Research Agreement Allocation Protocol)– OGSA BYTE IO WG– OGSA D WG (Data)– OGSA DMI WG (Data Movement Interface)


OMII-Europe Guiding Principles

• Committed to standards process– Implementing established open standards– Providing feedback to the standards process (e.g. OGF)

• Quality Assurance– Published methodology and compliance test– All software components have public QA process and audit trail

• Impartiality– OMII-Europe is “honest broker” providing impartial

advice/information on e-infrastructures


The Virtuous Cycle – Technology transfer with Grid projects and standards organisations

Globus

OMII-UK

CROWN

Components

Components

IN

OUT

JRA1

SA2

JRA4

JRA3

SA1

SA3

JRA2New Components

Standards Implementation

Standards Compliance Testing and QA

Benchmarking

Integrated Components

Supported Components on Eval. Infrastructure

Repository


OMII-Europe project scope

• Initial focus on providing common interfaces and integration of major Grid software infrastructures

• Common interoperable services:– Database Access– Virtual Organisation Management– Accounting– Job Submission and Job Monitoring

• Infrastructure integration– Initial gLite/UNICORE/Globus interoperability– Interoperable security framework– Access these infrastructure services through a portal


Repository of Open-Source Software

• Make available software reengineered within OMII-Europe and contributed by third parties– Single services/tools & complete distributions

• Leverage existing infrastructure & projects– ETICS

• Capture build & test configuration data for repeatability– NMI Build & Test Framework

• Manage cross-platform environment for build & tests– Condor

• Underlying execution infrastructure• Support and training was available


Standards status: May 2008• Accounting

– SGAS/DGAS/UNICORE-RUS• OGSA-Resource Usage Service (RUS – OGF draft specification) - implies: • Usage Record Format (UR) (OGF recommendation status awaiting implementation)

• Job Submission and Job monitoring– CREAM-BES, GLOBUS-BES, UNICORE-BES

• OGSA-Basic Execution Service (BES) version 1.0 (OGF final specification) – implies:• Job Submission Description Language (JSDL) version 1.0 (OGF final specification)

• Database Access– OGSA-DAI

• WS-DAIX and WS-DAIR are being implemented in OGSA-DAI (both currently OGF candidate standards at 1.0 awaiting implementation)

• Virtual Organisation Membership Service– VOMS implements the following:

• Security Assertion Markup Language (SAML) Version 2.0 OASIS (March ’05)• Extensible Access Control Markup Language (XACML) version 2.0 OASIS (February ’05)

• Information modelling– GLUE 2.0

• standard is currently draft, expected to go out for public comments by the end of April 2008, final version by September 2008


New Services:a Community-agreed Information Model for Computing Resources• OGSA-BES and JSDL are already considered by OMII-Europe

– lacking a common description of Grid resources suitable for discovery, monitoring and scheduling

• Working on the definition of next-generation GLUE Information Model in the context of OGF GLUE WG and its implementation

• Grid components MUST be instrumented to expose GLUE 2.0-based description which is

• Conformant with the spec• Conformant with the renderings

• GLUEMan:– WBEM-based framework for managing providers to produce information

according to the OGF GLUE 2.0 information model and to render them in different formats (XML, LDAP, SQL)

– designed and implemented to simplify and speed up the adoption of GLUE 2.0


18

Key Project Results• Outreach and training have created a high project profile

– OMII-Europe is seen as THE project delivering interoperability for grid middleware

– Presentations at conferences, conference booths, tutorials• Project has delivered a substantial set of common services across

multiple grid distributions – OGSA-DAI: GT4, UNICORE 6, gLite3, OMII-UK, CROWN– VOMS: GT4, UNICORE 6, gLite3– BES: GT4, UNICORE 6, gLite3, OMII-UK, CROWN– RUS: GT4, UNICORE 6, gLite3– GLUE2: UNICORE 6, gLite3 (both in development)

• Repository and QA activities provide a persistent resource of evaluated software services

• Integration activity continues to work with real end users– WISDOM, EU-IndiaGrid, EUFORIA, ...


Further Information

http://omii-europe.org

http://omii-europe.org/

building components for grid interoperability

Documents