www.eu-eela.eu e-science grid facility for europe and latin america eelas:the seed lgi / ngis:the...

www.eu-eela.eu

E-science grid facility forEurope and Latin America

EELAs: the seedLGI / NGIs: the goal

Bernard M. Marechal (EELA-2 Project Coordinator)CETA-CIEMAT & UFRJ (Spain & Brazil)

CONAIS 2009 Villahermosa (Mexico), 23.09.2009

Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 2www.eu-eela.eu

Outline

• Short EELA & EELA-2 story

• The EELA-2 model for long-term sustainability

• Proposal for future activities


Computationally intensive research



Virtual Organisations

E-Infrastructure, i.e.

DCI (Grid, Clouds,...)

Applications

Data

Instruments


The beginning

• Several meetings (Rio de Janeiro, CERN, Itacuruca) in 2004, foreseeing a EU–LA Grid Project supported by EC

• EELA proposal prepared, submitted and approved in 2005

• EELA worked hard and well during 2 years (2006 – 2007)

• EELA-2 prepared & submitted in 2007 and approved in 2008

• EELA-2 will end on 31 March 2010

• EC support: fade out in the future

• LGI / NGIs: fade in mandatory to support e-Science in LA on the long term


Projects in numbers

EELA-2 (CP-CSA under EU FP7) EELA (SSA under EU FP6)

E-infrastructure shared between Europe and Latin America

• EC support: 1.7 M€

• CIEMAT extra support: 0.4 M€

• 10 Countries (3 in Europe)

• 2 International Organisations

• 20 Members (7 in Europe)

E-science grid facility for Europe and Latin America

• EC support: 2.1 M€

• CETA-CIEMAT extra funds: 0.3 M€

• Currently 16 Countries (11 in LA)

• 2 new countries (Panama & Uruguay)

• 1 International Organisation (CLARA)

• Currently 78 Members (62 in LA)

• 32 Institutions joining (31 in LA)

• Currently 13 JRUs (9 in LA)

• 4 new JRUs (3 in LA)

At the final review EELA was awarded the highest EC rank:

“Good to excellent project”


Countries and Resources Centres

France

Ireland

Italy

Portugal

Spain

ArgentinaBrazilChileColombia CLARA (International)CubaEcuadorMexicoPeruVenezuela

New

• Panama• Uruguay


Joint Research Units in EELA-2• A Joint Research Unit (JRU) is a partnership for e-Science Collaboration, with no no

legal statuslegal status as such, formed between entities of the same nationality and meeting the following conditions:

– scientific and economic unity (internal MoU in place);– lasting a certain length of time (not linked to a particular project);– recognised by a public authority (minister, secretary of state, etc.).

One of the major

successes o

f

EELA-2

Country # of JRU members in total # of JRU members in EELA-2

Argentina 7 7

Brazil 18 14

Chile 5 5

Colombia 9 5

Ecuador 5 5

France 33 3

Ireland 18 1

Italy 14 2

Mexico 9 8Panama 3 3

Peru 7 7

Portugal 5 3

Spain >50 7

Venezuela 4 4

No JRUs were established in LA

during EELA


EELA & EELA-2 objectives

EELA

• Build a bridge between consolidated e-Infrastructure initiatives in Europe and emerging ones in Latin American

• Create a collaboration network to deploy a large portfolio of scientific applications on a well supported Pilot Test-bed

• Care in parallel of the training in grid technologies and of the knowledge dissemination and outreach

EELA-2

• Provide an empowered Grid Facility with versatile services fulfilling application requirements

• Ensure production quality services

• Ensure the long term sustainability of the e-Infrastructure beyond the term of the project

• Expand the current EELA e-Infrastructure

• Look for new communities outside academia (Industry and Business)


More Numbers

EELA-2 EELA

At the end of the Project

• # Sites: 19 (13 in Production)

• # Cores: 2700

• Storage: 100 TB

• # Jobs: 430,000

• Scientific domains: 4

• # Applications: 18- 12 in production- 6 gridified

• Conferences: 3

• Workshops: 8

• Tutorials: 18

• Trained people: 733

• Grid Schools: 2

After 1 year

• # Sites: 22

• # Cores: 5800

• Storage: 190 TB

• # Jobs : 1,000,000

• Scientific domains: 9

• # Applications: 55- 12 in production- 15 Gridified

• Conferences: 1

• Workshops: 4

• Tutorials: 8

• Trained people: 158

• Grid Schools: 1


810 participant · day (as of end on April 2009)

Participant · day

600 by year 2

Project goals DoW

300 by year 1

EELA-2 achievements - NA2

NA2 = Dissemination & Training

Much more training effort delivered than pledged



GILDA training-infrastructure

TutorialsGrid Schools

Support system Wiki pages /

EOC website

Gridification

Weeks

Gridification team

NA3 = Application SupportThe EELA-2 “gridification universe”

Villahermosa (Mexico) – CONAIS' 09 - 23.09.2009 14www.eu-eela.eu 14

2

Applications distribution per country, continent and scientific domain (Y1)



Certification Authority

Registration Authority

CA Applicant

Catch All Certification Authority

EELA-2 achievements - SA1

SA1 = Infrastructure ServicesEvery EELA-2 country has access to the LA PKI


Legacy Sitesprod.vo SitesOurGrid Sites

Resource Centres Map (Y1)



About 1 million jobs executed




ResourceCentres


Infrastructure Usage - all VOs (Y1)


Deployment of a sustainable perfSONAR-based multi-domain network monitoring service – 3 measurement points on EELA-2 RCs (Overlay network monitoring) already installed– Integration with available monitoring services in the domains (LA and EU-NRENs,

RedCLARA, GÉANT2, EGEE-III, etc.)

SA2 = Network Resource ProvisionEELA-2 Network Monitoring

LA-NREN A - MPs

RedCLARA MPs

EELA-2 Measurement Points (MPs)

LA-NREN B - MPs

LA-NREN

A

RedCLARA

LA-NREN

B

EELA-2MP A

EELA-2MP B

Visualization tool

Overlay network monitoring

Multi-domain monitoring



EELA-2 achievements - JRA1

JRA1 = Development of Services for Applications and Infrastructure

Activity Goals

• Making the e-Infrastructure more useful and wide spread to increase its reach …

Execution of grid middleware on top of different platforms, including Microsoft Windows Scavenging of idle resources, particularly in shared desktops

… and its usability New application-oriented grid services Support for the integration of a lightweight opportunistic grid middleware (OurGrid)

whose development is led by one of the consortium members

• With the effect of helping in fostering the sustainability of the e-Infrastructure


• Interoperability between OurGrid and gLite, planned for M16, just available (see “Hot News” in the EELA-2 Website homepage):

– MoU signed with other projects working on similar goals (EDGeS and SEE-GRID-SCI)

– Initial assessment of the possibility of using EDGeS G3 indicates the feasibility of using technology that has already been successfully used in a similar setting

– Initial evaluation of the innovative co-existence approach reveals advantages that had not been foreseen at the time the project was proposed

• Possibility of draining BoT applications out of the service grid, improving the performance of tightly-coupled applications

EELA-2 achievements - JRA1

Highlights of the 1st Year


The “global” Grid coverage

How to take care of the future of e-infrastructures?


Grid “Ecosystem” evolution in Europe

Testbeds Utility ServiceRoutine Usage

National

Global

Ref.: D

. Kra

nzlm

ülle

r pre

sem

tatio

n a

t EG

EE’0

8


…and in Latin America ?

EELA

EELA-2

?

4/2008-3/20101/2006-12/2007

4/2010 →


Some considerations…

24

• EELA-2 will end up on 01/04/2010

• Will 6-year of successful efforts to: – Set up a production quality Latin American e-Infrastructure at the level of

existing European Grids

– Acquire the various Grid technology skills to autonomously support the Latin American e-Infrastructure on the long term (about 1000 LA people trained and 10’s of local tutor formed)

– Allow LA scientific communities to: Strengthen their active participation in EU – LA collaborations by

satisfying the required contributions to computing resources

Enter new international collaborations to take part in new research field or attract new partners in their own field of research

– Altogether “contribute to amplify the relevance of e-Infrastructures, blazing the trail towards Latin American e-Science Initiatives via National Grid Initiatives (NGIs) created in cooperation with existing NRENs and RedCLARA” (EELA-2 proposal)

just be wasted ???


The EELA-2 proposal

25

• A 3-level organisation is proposed: – Local (RCs ↔ LAN/MAN)– National (NGIs ↔ NRENs)– Continental (LGI ↔ CLARA)

• Multi-middleware (gLite, OurGrid, OSG, …) structures: NGIs and LGI must not be middleware dependent

• Close synergy with EGI that foresees to support three middleware (ARC, gLite and UNICORE)

• Important role of both the LA NRENs and CLARA: fully justified by their participation in EELA and EELA-2

• Even more: e-Infrastructures (Grid) should be part of the NRENs and CLARA structure (cost effective, technically correct, politically easier)


The Latin American Grid Initiative (LGI)(http://documents.eu-eela.org/record/1119/files/)

RC: Resource Centre, NGI: National Grid Initiative, GOC: Grid Operation Centre, NREN: National Research & Education Network, NOC: Network Operation Centre, LGI: Latin American Grid Initiative, CLARA: Cooperación Latino Americana de Redes Avanzadas, GSC: Grid Support Centre, NSC: Network Support Centre


The Latin American Grid Infrastructure - GridLGI

• GridLGI is composed, from bottom to top of three layers, each of them including Grid and Network components:

– At the lower Institute level: The Resource Centre (RC) is the actual site of the Grid computing resources; The backbone LAN or MAN network connects the Computing and Storage Elements (CE &

SE) at the RC site;

– At the intermediate national level: The National Grid Initiative (NGI) integrates all Grid Institutions in the country and federates

under its Grid Operation Centre (GOC) all their RCs in a National Grid (GridNGI); The underlying National Research & Education Network (NREN) links country-wide the

Research and Education Institutes and optimizes / monitors their network activity via its associated Network Operation Centre (NOC);

– At the upper international layer: The Latin American Grid Initiative (LGI) which brings together all NGIs and supervises the

various GridNGIs operation from its Grid Support Centre (GSC); CLARA: it is the WAN network which integrates the Latin American NRENs and supervises

the overall RedCLARA network activity by means of its Network Support Centre (NSC).


RC tasks

RC

StorageStorageElement(s)Element(s)

ComputingComputingElement(s)Element(s)

1st Level1st LevelUser Supp.User Supp.

Reg. Auth.Reg. Auth.

MonitoringMonitoring

AccountingAccounting

SecuritySecurityinc. resp.inc. resp.


GOC tasks

GOC



M/W M/W Repos.Repos.

Cert. Auth.Cert. Auth.

VOMSVOMS

2nd Level2nd Leveluser supp.user supp.

DataDataCatalogsCatalogs

Info. Syst.Info. Syst.

L&BL&BWMSWMS


GSC tasks

GSC

M/W releasesM/W releases



Catch-all CACatch-all CA

Global VOMSGlobal VOMS

3rd Level3rd Leveluser supp.user supp.

Gen. Info. Gen. Info. SystemSystem

Knowledge repo.Knowledge repo.

High levelHigh levelGrid servicesGrid services


NSC tasks

Single point Single point of accessof access

NSC

Network quality Network quality indicatorsindicators

User ticket GSC TicketSystem

Ticket Processing

Network

Operations

Training and Dissemination

Grid Site Deployment

Middleware

...

NSC Team

RC 1

Grid Site 1

RC 2

Grid Site 2

LA-NREN A EU-NREN B

RedCLARA

Operated byNOC of NREN A

Operated byNOC of NREN B

Operated byCLARA

Operated byNOC of

Campus Network

Operated byNOC of

Campus Network

NSC

GÉANT2

Operated byDANTE

E2E servicesE2E services


Training & dissemination tasks

T&D

ConferencesConferences

DisseminationDisseminationMaterialMaterial

ConsultancyConsultancy

Event calendarEvent calendar

Training eventsTraining events


User support tasks

UserSupp.

GuidelinesGuidelines

NewNewcommunitiescommunities

Tech. requirementsTech. requirementsgatheringgathering

Local/remote supportLocal/remote support

Apps. Apps. RegistryRegistry


LGI Management / Governance


Possible structure of a Latin American NGI

A strong liaison with the NREN is strongly suggested/recommended.A strong liaison with the NREN is strongly suggested/recommended.Local conditions permitting, the NGI should Local conditions permitting, the NGI should

be incorporated into the NREN.be incorporated into the NREN.


Long-term sustainability status (1/2)

• Long-term sustainability of e-Infrastructures world wide is key for a continued support of scientific communities

• EELA-2 is definitely contributing to the establishment of JRUs and NGIs in Latin America, in close synergy with NRENs and CLARA, also in countries not initially involved in the project

• EELA-2 has defined a detailed model and architecture of the Latin American Grid Initiative

• Contacts with CLARA have formally been established and a Joint Committee has been formed at the Bogotá Conference (with Spanish participation) to analyse, agree on, and start implementing the proposed model


Clear support of the former CLARA President (C. Casasus)

Long-term sustainability status (2/2)


What’s round the corner? GISELA?

• No EELA-3 foreseen then …

• Grid Initiatives for e-Science virtual communities in Europe and Latin

America (GISELA)

• Objective of the proposal: Following about 4 years of successful activities in

EELA and EELA-2, the objective of the GISELA proposal is to guarantee the long-

term sustainability of the EU-LA e-Infrastructure to ensure the continuity and the

future enhancement of the European – Latin American Virtual Research

Communities. Building on the EELA-2 e-Infrastructure, its associated Services and

the numerous User Communities, the project will focus on two inter-related goals:

– Implement the NGI / LGI sustainability model, as specified in DSA1.3 (http://documents.eu-eela.org/record/1119/files/), in association with CLARA and collaborating with EGI.

– Provide the Virtual Research Communities with the suited e-Infrastructure and Application-related Services required to improve the effectiveness of their research.


GISELA summary proposal 1/3

• Call Identifier: FP7-INFRASTRUCTURES-2010-2

• URL: http://cordis.europa.eu/fp7/dc/index.cfm?fuseaction=UserSite.CapacitiesDetailsCallPage&call_id=263#infopack

• Funding Scheme: CP - CSA



• Work Packages and their objectives (short summary)

– NA1: Administrative and Technical Management of the Project EELA-2 type of Administrative and Technical management International Cooperation

– NA2: Dissemination and Outreach Organise Dissemination events focusing on Decision Makers interaction to get their support for

the LGI / NGI implementation Promote outreach actions (e.g. Conference talks, seminars, exhibitions, Virtual Days,..) to

widely spread the usefulness of e-Infrastructures for different research communities

– NA3: Virtual Communities Support Identify and support Virtual Research Communities (VRC) by fostering e-Science collaboration

among multidisciplinary groups Create and maintain support tools to ease the utilisation of the e-Infrastructure (Application´s

portals and database, self-training material, User’s and developer’s Guide, Help Desk support etc.)

Organise training activities – tailored to each VCR needs – to transfer knowledge concerning the e-Infrastructure utilisation

Validate the services provided by SA1, SA2 and JRA1 by performing tests with pilot applications



– SA1: NGI / LGI Infrastructure Services (as in DSA1.3) Deploy the catch-all Grid Operation Centre (GOC) as defined in DSA1.3 to serve as

model for future ROCs Implement the Grid Support Centre (GSC) as defined in DSA1.3 Support Virtual Research Environment (VRE) tools, systems and processes

– SA2: NGI / LGI Network Services (as in DSA1.3) Keep managing Network resources via the existing national Network Operation

Centres (NOC) Implement the Network Support Centre (NSC) as defined in EELA-2 DSA1.3 Proceed with the deployment of performing Network monitoring in collaboration with

EGI

– JRA1: Infrastructure & Application-oriented Services for Virtual Communities Increase the reach, the attractiveness, the usability and the effectiveness of the EU-LA

e-Infrastructure for well-established and new communities. For this goal, collaborate with EGI to divulge the Services developed in the EELA-2 context and devise those new Services required by the User Communities


Why is it possible?

• The EELAs – Trained users, tutors and administrators in LA– Disseminated the Grid culture– “Approached” Decision Makers– Proposed a model for long-term sustainability of Grid

infrastructures in LA– Is currently looking for funds in EU (EC & CIEMAT) to help

implementing the model during the next 2 years

ConclusionIt should work!!!!!!!!!!!!!!!!!!!!!


Thank you for your attention

[email protected]

www.eu-eela.eu e-science grid facility for europe and latin america eelas:the seed lgi / ngis:the...

Documents

eela slide

short eela eela

villahermosa mexico

marechal eela

objectives eela

numbers eela

ec eela proposal

final review eela