experience sharing from “euasiagrid” · (towards a common e-science infrastructure for the...

Experience Sharing from “EUAsiaGrid”

(Towards a common e-Science infrastructure for the European and Asian grids) and

“EGI-InSPIRE” (Integrated Sustainable Pan-European Infrastructure for Researchers in Europe)

Piyawut SRICHAIKUL, Ph.D. NECTEC, Thailand

EU FP7 Information Day

Opportunities for Participation by Thai Researchers

Thursday, June 23, 2011 Holiday Inn Bangkok Silom

THAILAND

2

Time Scale and Budget

• Coordination Support Action funding scheme

• Start date: April 1st, 2008

• Duration: 24 months (extend to 27 months, with partners added)

• Budget: 1.437k Euro

• EU contribution: 920k Euro

• Collaboration with EGEE-III, EU-IndiaGrid, EUChinaGRID and EGI_DS

• EUAsiaGrid is expected to run on Asia-Pacific infrastructure

• Status: Complete. Final review in Brussels, Aug 2010. Overall Assessment “Excellent progress”

3

PartnersPartners

INFN INFN ((ITIT))

CESNET CESNET ((CZCZ))

NCeSSNCeSS ((UKUK))

HealthGridHealthGrid ((FRFR))

AdMUAdMU ((PHPH))

ANU ANU ((AUAU))

ASGC ASGC ((TWTW))

ASTI ASTI ((PHPH))

HAII HAII ((THTH))

IDA IDA ((SGSG))

IAMI IAMI ((VNVN))

ITB ITB ((IDID))

NECTEC NECTEC ((THTH) )

UPM UPM ((MYMY))

MIMOS MIMOS ((MYMY))

NUS NUS ((SGSG))

IFI IFI ((VNVN))

Intersect Australia Ltd Intersect Australia Ltd ((AUAU))

4

Goal and Actions

• The project’s main goal will be to pave the way towards an Asian e-Science Grid Infrastructure

• Taking advantage of the existing global Grid technologies, with the specific emphasis on the European experience with the gLite middleware and applications running on top of it

• Its main actions will be to spread dissemination, provide training, support scientific applications and monitor the results

5

Breakdown in work packagesBreakdown in work packages

Work Package Title WP Coordinator

WP1 Project management INFN (M. Paganoni)

WP2 Requirement capture and coordination policy definition UNIMAN

(A. Voss)

WP3 Support of scientific applications CESNET (L. Matyska)

WP4 Dissemination ASGC (V. Huang)

WP5 Training INFN (M. Fargetta)

EuAsiaGrid Project • EU-FP7 Coordination and Support Action, 2008-

2010

• Partners: Four European + 9 AP + NECTEC, HAII

• Objectives

• Promote awareness, capture user requirements and build user community

• Integration with the wider Grid infrastructure and promote e-Science

• Five application areas:

• HEP

• Computational Chemistry

• Mitigation of natural disasters

• Bioinformatics and Biomedics

• Social Science Applications

• Survey of requirements

• Status, Plans and Requirements (SPR) document

• Scientific communities requirements capture and analysis

• Policy report / technical roadmap

EuAsiaGrid Project - NECTEC

• Contribute Resources

• UI, CE, DPM and MON machines

• two SE (one TB each)

• two worker nodes (each with 2 GHz Intel Xeon, 8 cores, 16 GB RAM) + one worker node (P4 2 GB RAM)

• NECTEC GOC CA

• User training for computational chemistry scientists

• Introduction to Grid computing

• g-Lite basic commands

• Gaussian / Turbomole job submission & management

• Applications

• Chemistry: AMBER, Autodock, GROMACS, TURBOMOLE, Gaussian

• High Energy Physics: CRAB tool, CMS VO

EuAsiaGrid Project-HAII • Contribute Resources

• UI, CE, MON, WMS machines

• SE (10 TB)

• 4 worker nodes (each with 2 GHz Intel Xeon, 12 cores, 4 GB RAM)

• Disaster Mitigation Workshop Series: BKK, Taipei

• Applications

• Grid for water security: reducing risks/damages from water related disaster in Southeast Asia

• Image Processing: observation of change in average sea surface temperature (SST) and sea surface height anomalies (SSHA)

• Climate Change: WRF – Typhoon forecast, Weather forecast (Wind Map)

www.egi.eu EGI-InSPIRE RI-261323

EGI-InSPIRE

www.egi.eu EGI-InSPIRE RI-261323 30/05/2011 Project Presentation - May 2011

EGI-InSPIRE Project Presentation

Steven Newhouse

Project Director, EGI.eu

30/05/2011 Project Presentation - May 2011 9

30/05/2011 Project Presentation - May 2011


Abbreviations

• EGI: European Grid Infrastructure • EGI.eu: European Grid Initiative organisation • EIRO: European International Research Organisation • ESFRI: European Strategy Forum on Research Infrastructures

• HUC: Heavy User Community • NGI: National Grid Infrastructure/Initiative • RP: Resource infrastructure Provider • SSC: Specialised Support Centre • UMD: Unified Middleware Distribution • VO: Virtual Organisation • VRC: Virtual Research Community



EGI-InSPIRE


Why build a European Grid Infrastructure?




Infrastructure (Wikipedia)

Infrastructure is the basic physical and organisational structures needed for the operation of a society or enterprise, or the services and facilities necessary for an

economy to function

EGI provides a service infrastructure that exposes and helps coordinate a resource

infrastructure


The Enterprise is the European Research Area



What is a Grid?

• A grid consists of distributed resources controlled by separate organisations that be systematically used securely by users external to that organisation

• Resources can include:

– Commodity or HPC clusters

– Disk or tape storage

– Instruments

– Data Archives or Digital Libraries




C21: Digital Research

Project Presentation - May 2011

Extracting Knowledge

from the Data Deluge

14 30/05/2011



Digital Agenda for Europe

• Borderless Services

• Interoperability

• Supporting Innovation


Digital Agenda for Europe “Europe should also build its innovative

advantage in key areas through reinforced

e-Infrastructures (i.e. GEANT & EGI)”

“Europe should also build its innovative

advantage in key areas through reinforced

e-Infrastructures (i.e. GEANT & EGI)”

remove barriers to the free movement

of knowledge


A Virtuous Service Cycle

Project Presentation - May 2011 16

EGI.eu

National Resource Providers

Current User Communities

New User Communities

MoUs MoUs & OLAs

Technology

Providers

MoUs & SLAs

30/05/2011



EGI-InSPIRE Project

Integrated Sustainable Pan-European Infrastructure for Researchers in Europe

A 4 year project with €25M EC contribution

– Project cost €72M

– Total Effort ~€330M

– Effort: 9261PMs

Project Partners (50)

EGI.eu, 38 NGIs, 2 EIROs

Asia Pacific (9 partners) Un-Funded

Project Presentation - May 2011 17 30/05/2011

Funded



Project Objectives

• A sustainable production infrastructure

– Resource providers in Europe and worldwide

– With new technologies as they mature

• Support structured international research

– Sustain current domain specific services

– Attract new user communities (e.g. ESFRI)




Project Activities

• NA1: Project & Consortium Management – Project Office and Quality Assurance

• NA2: External Relations – Policy Development and Dissemination – Community Building Events

• NA3: User Community Coordination – EGI.eu and NGI support teams – Supporting Technical Services for Virtual Research Communities

• JRA1: Support for Operational Tools – Maintenance and Development – Support for new resources and their accounting




Project Activities

• SA1: Operation of the production infrastructure – Infrastructure oversight and quality control – Operational security – Operational Tools, monitoring & accounting – Helpdesk & Support teams (NGI & centrally) – Validation and integration of new technology

• SA2: Provisioning the Software Infrastructure – Definition of software coming from external projects – Validation of delivered software – Software repository and support tools

• SA3: Support for Heavy User Communities – Services & tools for all users of the infrastructure – Domain specific support for current heavy users




European Grid Infrastructure (April 2011 and yearly increase)


Logical CPUs (cores) -239,840 EGI (+24.9%)

-338,895 All

102 PB disk and 89 PB tape

Resource Centres -338 EGI -345 All (+6.8 %)

-96 supporting MPI (+6.8%)

Countries (+11.5%) -51 EGI -57 All (+18.75)



EGI Resource Providers and Operations Centres 1/2

• 22 National Operations Centres

– Armenia, Belarus, Bosnia and Herzegovina, Bulgaria, Cyprus, Croatia, Czech Republic, France, FYR of Macedonia, Germany, Georgia, Greece, Hungary, Israel, Italy, Montenegro, Poland, Romania, Serbia, Slovakia, Slovenia and Turkey

• 5 Federated Operations Centres (16 NGIs)

– IberGrid (Portugal and Spain)

– The Netherlands Federation (Belgium and The Netherlands)

– Russian Federation (Russia and Ukraine)

– NDGF Federation (Austria, Denmark, Estonia, Finland, Latvia, Lithuania, Norway and Sweden)

– United Kingdom/Ireland Federation




EGI Resource Providers and Operations Centres 2/2

• 1 EIRO Operations Centre: CERN

• 4 Non-European Operations Centres

– Asia Pacific Federation: Australia, China, India, Japan, Malaysia, Philippines, South Korea, Taiwan and Thailand

– Canada Federation: Canada and China

– GISELA Consortium (IGALC Federation): Argentina, Brazil, and Venezuela

– Latin America: Brazil, Chile, Colombia and Mexico




Contacts

• EGI.eu Director – [email protected]

• EGI.eu Operations Team – [email protected]

• EGI.eu User Community Support Team – [email protected]

• EGI.eu Policy Team – [email protected]

• EGI.eu Dissemination Team – [email protected]

• EGI.eu Secretariat – [email protected]


Managing Outcome

Thailand National e-Science Project

• Five-year project to provide national infrastructure for e-Science in Thailand

• Run as a consortium

• All members manage with their own budget

• Heads of all institutes are committee

• Total Budget: USD$3.7Million

• Use Virtual Organization (VO) to manage application areas

• Shared resource through Grid middleware, GLite.

Application areas • Particle Physics Consortium

• Particle Accelerator, CMS, Standard & Beyond Standard models, etc.

• Climate Change Consortium

• RAMS, WRF, Sea Surface Temperature Anomalies, etc.

• Water Resource, Energy and Environment Management Consortium

• Flooding, Wind, Logistics, Strom Surge, etc.

• Computational Science and Engineering Consortium

• Solar Cell, CFD, Finite Element, Nano science, etc.

• Computer Science and Engineering Consortium

• HPC, Cloud Computing, 3D Visualization, Parallel Optimization etc.

Grid Infrastructure and Middleware

Partic

le Physics

Computer

Science and Engineering

Clim

ate C

hange

Water R

esource, E

nergy

and Environment

Computational

Science and Engineering

Current Founding Members

• National Electronics and Computer Technology Center

• National Nanotechnology Center

• Hydro and Agro Informatics Institute

• Chulalongkorn University

• Suranaree University of Technology

• King Mongkut’s University of Technology Thonburi

Lessons Learned-Observation

• EU FP7 is very European Centric by nature. Therefore, it is very difficult to identify the common ground to generate win-win value for research cooperation between European-Asian partners. The larger the development gap, the more significant of this issue. The collaborative project needs to reflect on this issue and find practical solution.

• From 1st hand experience, Asian partner need to secure adequate amount of resource from its organization to complete the project obligation and ensure value benefit to its country or for local benefit.

• There are many new terms and acronyms that Asian partner are not familiar with. It takes significant effort to understand them as well as procedures and regulations, enough to function.

• Cool project ACRONYM counts (a lot).

• A project presentation template: a must have item ☺☺☺☺

30

Lessons Learned-Observation

• EU FP7 program has very well structured project documentation and project assessment procedures. A very valuable experience.

• The project evaluation is strictly based on what you have “promised” and “delivered”.

• Communication skills (and open opinion) are really needed than ever.

• The project leader is the “key success factor”: Experienced, committed, highly skilled project leader is crucial.

• There are always high administrative cost (time).

• Social Network has strong influence in getting Asian partners into the project.

• A past successful project create opportunity for a future project.

31

Lessons Learned-Benefit

• Generally, you will learn new technical stuff to catch up the development gap.

• You will benefit a lot from being forced to take in professional teamwork environment with unfamiliar people and cultural diversity.

• Expanding your international recognition i.e. meeting new Asian partners and possibility to work with EU partners.

• Directly observed and experienced the required skills for international cross-cultural collaboration (leadership and management)

32

Lessons Learned-Advise

• Build your role into the project out of your enthusiasm and your interest. You are not in it for the money. EC funding will not cover every item that you need to do. The fact is that, you may have to put in as much or more than the EU contribution to run activities of your responsibility. Consider EU contribution as partial funding to help you to run your research.

• Identify common value: Your project deliverables must be common and committed goals of partners. Small number of outputs that have significant scale is far better than large number of small isolated outputs.

• Balance your benefit in various dimensions, not just monetary value.

• “Must take into account” of high administration cost into your work plan and resource.

33

Lessons Learned-Advise

• Do NOT try to lead the project in your first time around.

• It takes a friend to introduce friends. It’s a good way to start.

• Make sure that you spent enough time through the proposal development stage and understand your role, action items, and deliverables in the project before you commit your organization and your time to the project.

• During execution, it is important to have communication with the project leader and WP leader regularly. It is good for everyone.

• SPEAK UP. Unsolved issue will definitely come back to collect.

• It is better to have your IAF and your organization boss informed or involved from the beginning.

experience sharing from “euasiagrid” · (towards a common e-science infrastructure for the...

Documents