Pacific Rim Application and Grid Middleware Assembly - PRAGMA:

An Overview of Past and FutureCatalyzing Global Team Science

PRIME, PRIUS, GLEON

Peter ArzbergerPhilip Papadopoulos, Mason Katz

Gabriele Wienhausen, Linda FeldmanTim Kratz, Fang-Pang Lin

And many more

The Minority-Serving Institutions Cyberinfrastructure Institute MSICI226 June 2006

Why Bother with International Activities? • In the 21st century advances in science and engineering

(S&E) will, to a large measure, determine economic growth, quality of life, and the health of our planet. The conduct of science, intrinsically global, has become increasingly important to addressing critical global issues…Our participation in international S&E collaborations and partnerships is increasingly important as a means of keeping abreast of important new insights and discoveries in science and engineering [NSB 2000]

• What nations don’t know can hurt them. The stakes involved in study abroad are that simple, that straightforward, and that important. For their own future and that of the nation, college graduates today must be internationally competent. [Lincoln Report 2005]

• Peace and prosperity around the world depend on increasing the capacity of people to think and work on a global and intercultural basis. As technology opens borders, educational and professional exchange opens minds. [Annual Report IIE 2005, and http://www.iie.org/ “About”)

e-science’s New Frontier: Merging of Science and Information

Technology – PRAGMA’s Activities

PreviouslyUnobtainable

Observations andUnderstanding Enabling

Technology

• Advance science

Science Drivers

• Focus development

PersistentInfrastructure

• Broaden impact

Education & Capacity

Building

• Develop human resources

Sustained Collaboration

• Build teams and trust

PRAGMA’s Founding Motivations

• The grid is transforming e-science: computing, data, and collaboration

• The problem remains that the grid is too hard to use on a routine basis

• Middleware software and people need to interoperate

• Science is an intrinsically global activity

http://www.pragma-grid.net

Establish sustained collaborations

and

Advance the use of the grid technologies for applications

among a community of investigators working with leading institutions around the Pacific

Rim

Overarching GoalsPRAGMA

Working closely with established activities that promote grid activities or the underlying infrastructure,

both in the Pacific Rim and globally.

PRAGMA’s Role in the Community

• Constructed conduit of technology, ideas, information, people– Multi-directional flow among institutions and

communities of software, approaches

• Developed framework for collaboration– Catalytic in forming teams– Foundation for continued and greater work

• Built trust among members– Interoperability essential among people– Sharing of resources, ideas, and people resulted

Result: many unplanned successes

Collaborative Frameworkfor Success

• SARS: Grid Community Pulls together to Battle SARS

• GLEON: Launching new community effort– Building on EcoGrid in Taiwan

• Expanding Reach of Projects– Optiputer; NEESit; iGEON; IVOA

• PRIME: Creating opportunities for undergraduate students– Reciprocal Projects from Jilin, Osaka

• PRIUS: Osaka University, international internships and “PRAGMA Classes”

• KRocks: Localization of Rocks in Korea

Successes that happened because PRAGMA exists

Conduit for Success• Telescience: Sharing and Contributing Technology• Multi-way Dissemination, Integration, and Synthesis

of Software: – Rocks Rolls of SCE, Gfarm, Ninf-g– Ninf-G into NMI 8.0 (and soon Naregi CA) – iGAP/Gfarm, GAMESS/Nimrod

• Creation of Laboratory for Routine Use Experiments– Grew from 8 sites to 19 sites; Multiple, persistent experiments– Use and co-development of SCMSWeb and other software in

PRAGMA (e.g. CICESE porting SCMSWeb to solaris system; CNIC porting it to Itanium) and use of MOGAS (Grid Acct System – from PRAGMA 8)

– Joint papers Issues and Methods for Building a Multi-Application International Grid Resource (6 institutions on author list); Applications

• Assisted in RPC standards at GGF, and launching of International Grid Trust Federation (Lead by AIST)

Outcomes through on-going meetings

Overview and ApproachProcess to Promote Routine Use Team Science

Application-Driven CollaborationsApplications Middleware

Routine Use Lab/TestbedTesting Applications

Building Grid and GOC

Multiway DisseminationKey Middleware

Workshops and Organization

Information Exchange

Planning and Review

New Collaborations

New Members

Expand Users

Expand Impact

Products

Improved middlewareBroader Use

New CollaborationsTransfer Tech.

StandardsPublications

New KnowledgeData AccessEducation

PRAGMA Grid TestbedPRAGMA Grid Testbed

AIST, JapanCNIC, China

KISTI, Korea

ASCC, Taiwan

NCHC, TaiwanUoHyd, India

MU, Australia

BII, Singapore

KU, Thailand

USM, Malaysia

NCSA, USA

SDSC, USA

CICESE, Mexico

UNAM, Mexico

UChile, Chile

TITECH, Japan

QUT, Australia

UZurich, Switzerland

JLU, China

NGO, Singapore

MIMOS, Malaysia

OSAKAU, Japan

IOIT-HCM, Vietnam

http://goc.pragma-grid.net

Source: Cindy Zheng

PRAGMA Grid resourceshttp://goc.pragma-grid.net/pragma-doc/resources.html

Applications http://goc.pragma-

grid.net• Real science, multiple applications

– Resource sharing• Mpich-g2• Reservation and meta-scheduling

– TDDFT: quantum-chemistry, AIST, Japan– Savannah: climate Model, MU, Australia– QM-MD: quantum-mechanic, AIST, Japan– iGAP: bioinformatic, UCSD, USA– Gamess-APBS: organic chemistry, UZurich,

Switzerland– Siesta: molecular simulation, UZurich,

Switzerland– Amber: molecular simulation, USM, Malaysia– FMO: quantum-mechanics, AIST, Japan– HPM: Genomics, IOIT-HCM, Vietnam– (GEON, Sensor, … <data, sensor>)

SCMSWebhttp://www.opensce.org/components/SCMSWeb

• Web-based monitoring system for clusters and grid– System usage– Performance metrics

• Reliability– Grid service monitoring– Spot problems at a glance

Software Layers

• Globus 2, 3, 4• GT4 pre-WS, 9 sites

• GT4 WS, 1• Moving requirements

iGAPhttp://pragma-goc.rocksclusters.org/applications/igap/igap.html

• Genome annotation pipeline

• Use CSF/Gfarm

• Driver: Wilfred Li (SDSC), Osamu Tatebe

(AIST), Xiaohui Wei (JLU)

• Ready on 7 sites:– AIST: Yoshio Tanaka, Yusuke Tanimura– ASCC: Hurng-Chun Lee, Mike Chiang– KISTI: Jysoo Lee, Jae-Hyuck Kwak – NCHC: Weicheng Huang, Chien-Lin Huang– NCSA: Radha Nandkumar, Tom Roney– SDSC: Mason Katz, Cindy Zheng– TITECH: Satoshi Matsuoka, Hitoshi Aoki

Ninf-G/NMI/Rocks Integration• Ninf-G-2.4/NMI

– Lead integrators: • Yoshio Tanaka (AIST)• Mats Rynge (NMI)

• Ninf-G 2.4 Rocks Roll – Lead integrators:

• Mason Katz (SDSC)• Yoshio Tanaka (AIST)

Ninf-G v2.4.0 released in NMI v8first experience for NMI to include non-U.S. software.

Evaluate and improve Ninf-G2 through the PRAGMA routine-basis experiments

GRID TECHNOLOGIES

PortalsMiddlewareGraphics

Computational Chemistry EngineData Analysis Tools

Hardware

ENABLING NEW SCIENCE

Exploiting grid technology & hybrid computational methods

PARAMETER SEARCH

4 variables15,876 points

Refineable hypersurface

ICCS’03 PRAGMA 4

Monash, Australia HPCC, Japan CRAY, Japan SDSC, USA UCSD, USA CPE, Thailand KISTI, Korea

GEOGRAPHIC DISTRIBUTION OF JOBS DURING EXECUTION

Source: Wibke Sudholt, Kim Baldridge, David Abramson, Colin Enticott, Slavisa Garic

GAMESS and Nimrod/G

The PRAGMA Steering Committee

http://www.pragma-grid.net/steering_committee.htm

Working Groups: Organize PRAGMA Efforts

• Resources– Mason Katz, SDSC– Yoshio Tanaka, AIST

• Biological Sciences– Karpjoo Jeong, Konkuk

U/KISTI

• Telescience– Shinji Shimojo, Osaka– Fang-Pang Lin, NCHC

• Data Computing– Osamu Tatebe, AIST

Some Members and Participants

http://pragma-goc.rocksclusters.org/pragma-doc/org.html; Resource Group Album: Total 85 Pictures – and another album

PRAGMA Future Meetings• PRAGMA 10

– QPSF, JCU, APAC, Townsville, Australia 26 – 28 March 2006

– Bringing the Grid to coastal zones; Held in conjunction with GLEON and Coral Reef

• PRAGMA 11– Osaka University, Japan, approx. 15 – 17 October

2006– Preparing Future Generations; in conjunction with

PRIUS program• PRAGMA 12

– NECTEC, Kasetsart University, Thailand, Spring 2007– Advancing Collaborations with ThaiGrid

• PRAGMA 13– NCSA, Illinois, USA, Fall 2007– PRAGMA Engagements in Cyberenvironments

• PRAGMA 14– NCHC, Taiwan, Spring 2008– Living Grids; Held in conjunction with Taiwan Grid

Activities

Annual Reports

•Prepare for SCxy•Highlight Accomplishments•Promote PRIME and PRIUS•Describe Working Group Progress•Summarize Individual Institution Contributions•Give Additional Information, e.g. Publications, Sponsors

Articles On• Infrastructure in

Australia

• India’s Grid Initiative

• Cyber Science Initiative in Japan

• CI in Korea

• Taiwan CI for Knowledge Innovation

• Brazil

• South Africa

• PRAGMA

http://www.ctwatch.org/quarterly/articles/2006/02

Guest Editor: Radha Nandkumar

CCGrid - Singapore16 – 19 May 2006

• Abramson D, Lynch A, Takemiya H, Tanimura Y, Date S, Nakamura H, Jeong K, Hwang S, Zhu J, Lu ZH, Amoreira C, Baldridge K, Lee H, Wang C, Shih HL, Molina T, Li, W, Arzberger P. Deploying Scientific Application on the PRAGMA Grid Testbed: Ways, Means and Lessons. CCgrid 2006

• Lee B-S, Tang M, Zhang J, Soon OY, Zheng C, Arzberger P. Analysis of Jobs on a Multi-Organizational Grid Test-bed. CCGrid 2006.

• Huang W, Huang C-L, Wu, C-H., The Development of a Computational Grid Portal. Accepted CCGrid 2006.

• Zheng C, Abramson D, Arzberger P, Ayuub S, Enticott C, Garic S, Katz M, Kwak J, Papadopoulos P, Phatanapherom S, Sriprayoonsakul S, Tanaka Y, Tanimura Y, Tatebe O, Uthayopas P. The PRAGMA Testbed: Building a Multi-Application International Grid CCGrid 2006.

Preparing the Global Workforce

PRIME: Providing Students International Interdisciplinary Research Internships and Cultural Experiences

preparing the global workplace of the 21st century•Computer Network Information Center (CNIC), Chinese Academy of Sciences•Cybermedia Center (CMC), Osaka University, Japan•Monash University, Australia•National Center for High-performance Computing (NCHC), Taiwan

PRIME 2005 – Presentations at iGRID 2005• Phylogeny Determined by Incomplete Protein Domain

Content, I.Lee, CNIC• A visualization of network measurements, J.Lee, CNIC• Deployment and Extension of JuxtaView for the Scalable

Adaptive Graphics Environment, C.Cheung, NCHC• Developing the Interface between PDA and Sensors, O

Langman, NCHC* (from U Wisconsin)• Visualizing internet connectivity using Cytoscape, S.Lee,

NCHC• Extending EcoGrid Capability, D.Leu, NCHC*• BOINC as a Nimrod Resource for Quantum Chemistry,

J.Hwang, Monash• Computational Grid Tools for Protein-Ligand Docking

Studies, L Berstis, Monash• Modeling Cardiac Rhythm Alternation, J.Nevo, Monash• SNPs, Protein Structure and Disease, D.Bitton, Monash• Computational Cardiac Modeling, D.Dederko, Monash• The Development of A Querying System for Structured

Metadata in a Datagrid Environment, J.Chen, Osaka• Visualization Tools for Bio-molecular Simulation, C.Liang,

Osaka• A Bio-molecular Simulation Portal, E.Wang, Osaka

prime.ucsd.edu/presentations SC’05

Background and Motivation of PRIUS Pacific Rim International UniverSity

• Success of PRAGMA community– Accumulated Expertise and practice of building Highly advanced

Grid applications and middleware– Formation of Human network of researchers and scientists in

pacific rim

• Activation of Educational activities– PRIME: Providing UCSD students with an opportunity of experiencing,

studying, and learning highly sophisticated technology and cultural background through practical R&D with PRAGMA partners.

Maturity of PRAGMA R&D networkMaturity of PRAGMA R&D network

Stimulating the movement toward Stimulating the movement toward the establishment of PRAGMA Educational networkthe establishment of PRAGMA Educational network

PRIUS: Pacific Rim International UniverSities12 Oct 05

•

• Exchange among PRAGMA Sites• Lectures from PRAMGA members

PRAGMA 11Oct 2006 –

to expand PRIUS

Rough Approach of PRIUS

• Establish a consistent educational program for graduate students.

• On-the-Job-Training Education on PRAGMA

Lecture course:Stimulate students’ ambitious

Short Abroad Internship:Offer trial opportunities

Long Abroad Internship:Provide skill building chances

M1

M2

Doc

P.D Exchange prgm: produces liquidity in human resource

P.D

Educational Part

Research Part

Sou

rce

: S

usum

u D

ate

, O

saka

Uni

vers

ity

http://prius.ics.es.osaka-u.ac.jp/en/index.html

Taiwan’s Natural Beauty

Lake Metabolism Website

http://lakemetabolism.org

10

12

14

16

18

20

22

20-Aug-04

21-Aug-04

22-Aug-04

23-Aug-04

24-Aug-04

25-Aug-04

26-Aug-04

27-Aug-04

28-Aug-04

29-Aug-04

30-Aug-04

31-Aug-04

Wat

er T

emp

erat

ure

(C

)

0

5

10

15

20

25

30

35

40

Win

d S

pee

d (

m/s

)

Surface Temp (C)

0.5m Temp (C)

1m Temp (C)

1.5m Temp (C)

2m Temp (C)

2.5m Temp (C)

3m Temp (C)

Wind Speed (m/s)

RAIN_FALL

Wind Speed

Precipitation (mm/5 minutes)

Typhoon causes water column mixing

Mixing event

Source: Tim Kratz

From Cole, J. J., N. F. Caraco, G. W. Kling, and T. K. Kratz. 1994. Carbon dioxide supersaturation in the surface waters of lakes. Science 265:1568-1570

Of 4665 samples from 1835 lakes worldwide,87% were supersaturated

Why?

Source: Tim Kratz

Programs-Australia-Canada-China-Finland-Florida-New Zealand-Israel-South Korea-Taiwan-United Kingdom-Wisconsin

First meeting:San DiegoMarch 7-9, 2005

Source: T. Kratz

Steering Committee-Peter Arzberger, UCSD, USA-David Hamiltion, University of Waikato, New Zealand-Tim Kratz, University of Wisconsin, USA-Fang-Pang Lin, NCHC, Taiwan

Scalable instrumentation and cyberfrastructure is critical

We can do this scale nowhttp://lakemetabolism.org Source: Tim Kratz

Problematic, but possible with today’s cyberinfrastructureSo

urc

e: T

im K

ratz

Scale needed to answer regional/continental questions

Not currently possible

Source: Tim Kratz

New Paradigm: Global Team Science

U.WaikatoD.Hamilton

Models

NCHCF.P.Lin

Maintain YYLParallelize Codes

U.WisconsinT.Kratz

Maintain Trout BogLake MetabolismUCSD

F.Vernon, S.Peltier,T.Fountain P.ArzbergerROADNet, TelescienceMoore Fnd, PRAGMANIGLAS

B.Q QinMaintain Taihu

Physical Limnology

Kangwon UB.Kim

Maintain SoyangPublic Policy

e-science’s New Frontier: Merging of Science and Information Technology – PRAGMA’s and

GLEON’s Activities

PreviouslyUnobtainable

Observations andUnderstanding Enabling

Technology

• Advance science

Science Drivers

• Focus development

PersistentInfrastructure

• Broaden impact

Education & Capacity

Building

• Develop human resources

Sustained Collaboration

• Build teams and trust

•Bioscience•Chemistry•Envir .Sci.•Geosciences•Lakes (GLEON)•Others

•Ninf-G•Gfarm•Nimrod•SCMCWeb•MOGAS•KRocks•Others (Web service)•Testbed

•Links to resources

•PRAGMA &GLEON Framework

•PRIME•PRIUS•Others

International Education Matters

• Globalization is driving demand for an internationally competent workforce. Our nation's trade with Asia, now concentrated in California, has exceeded Europe since 1979 and is expected to approach $1 trillion a year this decade. The majority of future growth for industries of all sizes is in overseas markets; today one in six new jobs created flows from international trade.

• Access to good jobs will require new skills and competencies. Future careers in business, government, health care, law enforcement, and other good jobs will all require global knowledge and skills. Unfortunately, minorities are underrepresented in international careers and must be exposed to global content earlier in their education.

• Solving new national and human security challenges, including terrorism and HIV/AIDS, and the ongoing problems of poverty and environmental degradation, will require increased knowledge of other world regions, cultures and languages.

• Increased diversity in our nation's classrooms, workplaces, and communities, including new immigrants from many different parts of Asia and Latin America, requires greater understanding of the myriad cultures and histories students bring to school.

•http://www.internationaled.org/why.htm

Acknowledgements

• PRAGMA– Philip Papadopoulos (UCSD)– Mason Katz, Wilfred Li, Kim

Baldridge, Tomas Molina, Cindy Zheng

– Fang-Pang Lin (NCHC)– And many others at all 28

institutions, in particular the Steering Committee

• GLEON– Tim Kratz (U WI)– David Hamilton (U Waikato)– Fang-Pang Lin (NCHC)– And others at 10 other sites

• CREON– Sally Holbrook (UCSB)– Stuart Kininmonth (AIMS)

• PRIME– Gabriele Wienhausen– Linda Feldman– All Host sites and students

• PRIUS– Shinji Shimojo (Osaka)– Susumu Date (Osaka)

• CAMERA– Larry Smarr

• NSF– Bill Chang– Many others

• Gordon and Betty Moore Foundation

• National Institutes of Health

Additional Slides

e-science’s New Frontier: New Directions and Activities for PRAGMA

PreviouslyUnobtainable

Observations andUnderstanding Enabling

Technology

• Advance science

Science Drivers

• Focus development

PersistentInfrastructure

• Broaden impact

Education & Capacity

Building

• Develop human resources

Sustained Collaboration

• Build teams and trust

•Geosciences•Metaproteomics driver•Real-time sensing•Access to microscope•Coupling chem codes

•OptIPuter•Collab. Env. •GOC•Gateways•Globus•Planet Lab

•Links to production resources (TeraGrid, others) •Middleware interactions UK, Netherlands (Switzerland)

•Grids in China•SE Asia, NZ•Central/S. Am.

•Training

e-science’s New Frontier: Merging of Science and Information

Technology – PRAGMA’s Activities

PreviouslyUnobtainable

Observations andUnderstanding Enabling

Technology

• Advance science

Science Drivers

• Focus development

PersistentInfrastructure

• Broaden impact

Education & Capacity

Building

• Develop human resources

Sustained Collaboration

• Build teams and trust

•Bioscience•Chemistry•Envir .Sci.•Geosciences•Others

•Ninf-G•Gfarm•Nimrod•SCMCWeb•MOGAS•KRocks•Others (Web service)•Testbed, AMP

•Links to resources

•PRAGMA Framework

•PRIME•PRIUS•Others

e-science’s New Frontier: Merging of Science and Information

Technology – PRAGMA’s Activities

PreviouslyUnobtainable

Observations andUnderstanding Enabling

Technology

• Advance science

Science Drivers

• Focus development

PersistentInfrastructure

• Broaden impact

Education & Capacity

Building

• Develop human resources

Sustained Collaboration

• Build teams and trust

e-science’s Team Science: Merging of Science and Information

Technology

PreviouslyUnobtainable

Observations andUnderstanding

EnablingTechnology

• Advance science• Parameter Sweep• Dist.Data Files• Web Services• Cross-site query• Collaborative Tools• Many more

Science Drivers

• Focus development• Chemistry• Biology• Lakes• Geosciences• Many more

PersistentInfrastructure

• Broaden impact• Lambda Grids• Wireless sensor network

Education & Capacity

Building

• Develop human resources• Students and postdocs

Sustained Collaboration

• Build teams and trust• Many meetings