EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme1

EU-IndiaGrid Interoperability: status

and plans

Marco Verlato

International Grid Interoperability andInteroperation Workshop (IGIIW07)

Bangalore, 10 December 2007

EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme2

Outline

• EU-IndiaGrid project overview– Objectives– Activities– 1st year achievements

• Interoperability between EGEE and GARUDA– Analysis– Strategy– Some scenarios

• Conclusions

EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme3

An EGEE related project

EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme4

Indian Grid Projects• India has two main grid infrastructures:

– The GARUDA National Grid Initiative – The Department of Atomic Energy (DAE) Grid

IGCAR: wide-areaData dissemination

BARC: Computing with shared

controls

VECC: real-timeData collection

CAT: archival storage

4 MbpsLink 4 Mbps

Link

DAE Grid(gLite based)

GARUDA Grid•Private Network•Connecting 45 institutes in 17 cities•100 and 10 Mbps links

EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme5

GARUDA components

Resources

• Compute, Data, Storage,

• Scientific Instruments,

• Softwares,..

Access Methods

• Access Portal

• Problem Solving Environments

Data Management

• Storage Resource Broker

Development Environment

• DIViA for Grid

Management & Monitoring

• Paryaveekshanam

Collaborative Environment

• Video Conferencing over IP

• Access Grid

Resource Mgmt & Scheduling

• Moab from Cluster Resources

• Load Leveler, Torque

• Globus 2.x

Application (PoC)

• Disaster Management

• Bioinformatics

Slide from Natraj C Addekoppa’s talk at Taipei Workshop

EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme6

EU-IndiaGrid goals• To support the interconnection and

interoperability between EGEE and GARUDA for the benefit of e-Science applications

• To build an e-Science Network Community:– identify and aggregate research, scientific and industrial

communities which may benefit from the use of Grid

• To promote the use of advanced Grid technologies in scientific communities– Specific pilot applications (HEP, Biology, Material Science,

Earth and Atmospheric Sciences) and dissemination activities

• Project start: October 2006• Duration: 24 months

EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme7

PARTICIPANTSEUROPE• INFN (project coordinator) • Metaware SpA• Italian Academic and Research Network (GARR) • Cambridge University

INTERNATIONAL• Abdu Salam International Centre for Theoretical Physics

INDIA

• Indian Education and Research Network (ERNET) • University of Pune • SAHA Institute of Nuclear Physics, Kolkata• Centre for Development of Advanced Computing (C-DAC) • Bhabha Atomic Research Centre, Mumbai• TATA Institute for Fundamental Research (Mumbai)

National Centre for Biological Sciences (Bangalore)• Not formally, but concretely: VECC, Kolkata and MIT, Chennai

EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme8

Activities

WP1Project administrative and

technical management INFN+all

WP2 Building an eScience Network Community

MTW+GARR,ICTP,NCBS,ERNET, PUNE,

CDAC, BARC, TIFR

WP3 Network Planning & Support GARR, ERNET

WP4 Pilot grid infrastructure operational support

INFN+CAMBRIDGE, ERNET CDAC, BARC,

TIFR

WP5Applications

INFN+ICTP,CAMBRIDGE, NCBS, PUNE,SAHA, CDAC, TIFR

WP6 Dissemination & Networking Events

MTW+all

EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme9

Main 1st year achievements

• Five Project Workshops + several trainings– Mumbai, Dec 2006; Bangalore, Apr 2007; Kolkata, Jun

2007; Pune, Sep 2007; Taipei, Nov 2007

• EU-IndiaGrid Project Conference here this week• Enabling exploitation of high bandwidth EU-India

network connectivity– 45Mbps in October 2006; 1Gbps November 2007

• Supporting the creation of the National Indian CA • Operating the euindia VO and infrastructure with

>1200 CPUs , >10 CPU.years consumed by appl.s• many scientific applications ported on both

GARUDA and EGEE grids see posters at this conference

EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme10

Interoperability/1

• The 2nd year will be more focused on Interoperability

• The 1st year was spent in:– achieving some pre-requisites:

• ASGC CA acting as catch-all CA until the Indian National CA is established and approved by APGridPMA

• Interconnecting GARUDA’s private network with the intercontinental link

– carefully analysing the gLite and GARUDA middlewares– discussing among partners a number of possible

interoperability scenarios – defining a stepwise strategy to achieve the goal

EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme11

Interoperability/2• It was soon recognized the need of:

– taking advantage from EUChinaGrid experience about interoperability with CNGrid GOS middleware

– identifying solutions being compliant with consolidated/emerging standards (GLUE, JSDL, OGSA-BES, gridFTP, SRM)

• Corresponding actions undertaken:– EU-Asia Interoperability sessions/workshops/round tables

organised at international grid events• at EGEE’07 of Budapest, October 2007 • at WLCG-Tier2/EU-IndiaGrid Workshop of Taipei, November

2007• at IGIIW and EU-IndiaGrid Conference of Bangalore, this week

– Participation to Open Grid Forum Interoperability Panel planned from OGF22, February 2008

EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme12

Picture taken at Mumbai Workshop, December

2006

ANALYSIS

EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme13

MOAB

GLOBUS

lcg-CE(GT2 based)

GARUDA-CE(GT2 based)

BDII

GSI+VOMS

PORTAL UI

WMS

JC+LM

LFC

SRMSRB

MDS

GSI+gridmapfile

PBS-LikeLRMS

WN-1

WN-2

WN-nLinux + AIX

MW components analysis

EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme14

The strategy

• To deploy gLite middleware at Indian partners sites– CE and SE deployed at VECC, Kolkata in September 2007– First full-fledged gLite testbed deployed at MIT, Chennai in

October 2007– CDAC-Bangalore and Pune coming soon

• To implement a “basic” job submission interoperation scenario with the present middleware components

• To implement a more advanced scenario evaluating/adapting the EUChinaGrid Gateway

• To implement Data Management interoperation leveraging SRM-SRB interface developed by ASGC

• To design a true interoperability model based on OGF standards, to be implemented in a future I3 project

EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme15

gLite WMS

BDII

gLite UI

VOMS

Basic Job submission interoperation

GARUDA portal

gLiteResources

cloudWNs must supportboth environments

lcg-CE(Torque)

lcg-CElcg-CE(Torque)

lcg-CE(Torque)

In case of AIX sys.could be evaluatedAIX-ENEA solution

Mixed WN:GARUDA + gLite TAR-WN

gLite (TAR) UI or subset (WMproxy APIs)

EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme16

ENEA-Grid approach to provide access to AIX

Can be applied on other contexts, being based on:

1) availability of a shared file system on the site WN and on the GRID Proxy WN

2) use of a remote execution tool for implementing the wrapper

it works also withNFS or GPFS

it works also withrsh or ssh

Invasiveness of the grid middleware and firewall requirements are minimized !

EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme17

EUChinaGrid Gateway/Adapter Approach

Slides from Subrata Chattopadhyay’s talk at EGEE’07

EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme18

Data Management Interoperation

SRB

SRM• Short term plan– Implement interface

compatible with SRM version 2.2

• Mid-term plan– Interact and test with

other data management systems such as Castor, dCache,…,etc.

• Long-term plan– Interoperate with gLite.

• Hope glite users who join VO can access files and resources in SRB and vice versa.

• ASGC is developing a SRM interface

to SRB• DEMO at SC’07

Thanks to Fu-Ming Tsai

EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme19

Architecture Overview

Cache server (+gridftp server)

Core

Cache repository

SRB+DSI

File catalog

SRB/gridftp

Gridftp/management API

Gridftp/management API

SRM API

File transfer (gridftp)

File transfer (gridftp)

Web Service

Data server management

Users/applications

Thanks to Fu-Ming Tsai

• The ICTP team developing StoRM is part of both EU-IndiaGrid and SRM-WG of OGF

• Possible cooperation with ASGC for:– interacting with

StoRM – deploying/

adapting the SRM-SRB interface in EU-IndiaGrid

EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme20

Interoperability with OGF standards

• OMII-Europe EU project is delivering components to enable interoperability across different middlewares:

– gLite, Globus, UNICORE, OMII-UK and CROWN • For gLite-Globus we could use:

– CREAM-BES: new gLite CE compliant with BES and JSDL, beta release in December, stable release by April

– GT4 OGSA-BES: pre-alpha version GT4 CE– VOMS for the security (compliant with OGF OGSA AuthZ WG )– GLUE Schema for the Information System

• adapting these components to GARUDA and gLite environment requires effort for development JRAs in I3 EU Projects

EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme21

gLite WMS

JC+ LM

GARUDA Scheduler / MOAB

GLOBUS

lcg-CE(GT2 based)

GARUDA-CE(GT2 based)

GT2

gLiteBES conn.

globusBES conn.

BDII with GLUE schema

GSI +VOMS

GARUDAPortal gLite UI

Future job submission using JSDL/BES standards?(thanks to Moreno Marzolla / OMII-EUROPE)

GARUDA-CE(GT4 based)

BES

GT4 ICE

CREAM-CE(WS based)

BES

CREAM-CLI

BES-CLI

EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme22

• The main focus of EU-IndiaGrid Project in its 2nd year will be the interoperability between gLite and GARUDA

• Several discussions held, several approaches figured out during the 1st year, and a stepwise strategy has been proposed

• The Project is actively organizing panel sessions, round tables, discussions about interoperability with other Projects interested in this matter (EGEE’07 in Budapest, WLCG-WS in Taiwan, eScience 2007 in Bangalore)

• A contact with OGF was started late September 2007, the participation at Interoperability Panel at upcoming Open Grid Forum events is planned

• EU-IndiaGrid is ending in September 2008• A proposal is ready for EU-IndiaGrid2, describing a more

detailed way on standards adoption/collaboration for job and data management interoperability

Conclusions

EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme23

Thank you