cyberinfrastructure planning at nsf steve meacham national science foundation net@edu february 8,...
TRANSCRIPT
Cyberinfrastructure Planningat NSF
Steve Meacham
National Science Foundation
Net@EDU
February 8, 2006
Recent Happenings
• Office of Cyberinfrastructure established
• Search for Office Director
• Advisory Committee for CI
• Agency-wide Strategic Planning Process Underway
• Cyberinfrastructure Council established
Learning& WorkforceDevelopment
Data, Data Analysis &
Visualization
CollaboratoriesObservatories
& VirtualOrganizations
Cyberinfrastructure Components
High PerformanceComputing
NSF CI FY07 Budget Request
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Examples of FY07 Areas of Emphasis
• Leadership-class HPC system acquisition• Data- and collaboration-intensive software services• Confidentiality protection and user-friendly access for major
social and behavioral science data collections• National STEM Digital Library (NSDL) supporting learners at all
levels• CI-TEAM, preparing undergraduates, graduate students,
postdocs and faculty to use cyberinfrastructure in research and education
• Support for the Protein Data Bank (PDB), the international repository for information about the structure of biological macromolecules, and the Arctic Systems Sciences (ARCSS) Data Coordination Center
Strategic Plan(FY 2006 – 2010)
Ch. 1: Call to Action
Ch. 2: High Performance Computing
Ch. 3: Data, Data Analysis & Visualization
Ch. 5: Learning & Workforce Development
Ch. 4: Collaboratories, Observatories and Virtual Organizations
Strategic Plans for:
Completion in Spring, 2006 - Public comment soughthttp://www.nsf.gov/dir/index.jsp?org=OCI
Creating a World-Class HPC EnvironmentTo Enable Petascale Science and Engineering
Strategic Plan for High Performance Computing
(FY 2006 – 2010)
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.QuickTime™ and a
TIFF (LZW) decompressorare needed to see this picture.
Strategic Plan for High Performance Computing
(2006-2010)
Private Sector
Agency Partners
HPC Resource Providers
S&ECommunity
Portable, Scalable Applications Software &Services
SoftwareService
Provider (SSP)
SSP
SSP
Science-Driven HPC Systems
ComputeEngines
Local Storage Visualization
Facilities
Acquisition Strategy
FY06 FY10FY09FY08FY07
Sc
ien
ce a
nd
en
gin
eerin
g
cap
ab
ility
(log
rithm
ic s
cale)
Typical university HPC systems
Track 1 system(s)
Track 2 systems
HPC Capability By Center
Centers
Peak Teraflops
Xeon cluster
Alpha cluster
Cray XT3
BG/L x2
Altix
Power5
Power4 x2
Itanium2 clusters x2
Opteron clusters
NCSASDSC
NCARPSC
0
5
10
15
20
25
30
35Ag
greg
ate Teraflo
ps
Main HPC Center Resources
NCSA ResourcesHEC Resource Name Cobalt Tungten Mercury CopperVendor/Model SGI/Altix Dell IBM IBM p690Peak flops 6.6 TF/s 16.4 TF/s 10 TF/s 2 TF/sTotal number of procs 1024 2560 1282 384Architecture SMP/DSM Cluster Cluster SMP clusterProcessor manuf. Intel IA64 Intel IA32 Intel IA64 IBM POWER4Total memory/disc 3TB/370TB 3.8TB/122TB 4.6TB/230TB 1.5TB/35TB
SDSC ResourcesHEC Resource Name DataStar TeraGrid IntimidataVendor IBM/p655,p690 IBM IBM/BG/LPeak flops 15.8 TF/s 3.1 TF/s 5.7 TF/sTotal number of procs 2528 512 2048Architecture SMP Cluster Cluster MPPProcessor manuf. IBM POWER4+ Intel IA64 IBM PPC440Total memory/disc 7.3TB/126TB 1TB/400TB 0.5TB/20TB
PSC ResourcesHEC Resource Name TCS - Lemieux Rachel BigBenVendor HP HP CrayPeak flops 6 TF/s 0.3 TF/s 10 TF/sTotal number of procs 3000 128 2112Architecture MPP SMP MPPProcessor manuf. HP EV68 HP EV7 IBM PPC440Total memory/disc 3TB/70TB 0.5TB/12TB 2.1TB/200TB
NCAR ResourcesHEC Resource Name Bluesky Bluevista Frost LightningVendor IBM p690 IBM p575 IBM BG/L IBM e1350Peak flops 8.3 TF/s 4.7 TF/s 5.7 TF/s 1.1 TF/sTotal number of procs 1600 576 2048 260Architecture SMP Cluster SMP Cluster MPP SMP ClusterProcessor manuf. IBM POWER4 IBM POWER5 IBM PPC40 AMD OpteronTotal memory/disc 3.2TB/31TB 1.2TB/55TB 0.5TB/? 0.5TB/?
HPC Usage By Research Topic
HPC Usage by Disciplineat NSF-Funded HPC
20%
16%
13%12%
11%
10%
7%3%
1%2%
3%1%
1%
Atmos Sci
Biology
Astron/Astrophys
Chemistry
Physics
Chem Eng
Materials Sci
Earth Sci
Adv Sci Comp
Computer Sci
Oceanography
Elec Eng
OTHER
TeraGrid: What is It?
TeraGrid: (1) Provides a unified, user environment to
support high-capability, production-quality cyberinfrastructure services for science and engineering research.
(2) Provides new S&E opportunities – by making possible new ways of using distributed resources and services
Examples of services include:• HPC• Data collections• Visualization servers• Portals
• Integration of services provided by grid technologies
• Distributed, open architecture.• GIG responsible for integration:
• Software integration (including the common software stack, CTSS)• Base infrastructure (security, networking, and operations)• User support• Community engagement (including the Science Gateways activities)
• 8 Resource Providers (with separate awards):
• PSC, TACC, NCSA, SDSC, ORNL, Indiana, Purdue, Chicago/ANL• Several other institutions participate in TeraGrid as a sub-awardees of the GIG
• New sites may join as Resource Partners
Data CI: - Investments will continue to be prioritized by science and engineering research and education needs - S&E data generated with NSF funds will be accessible & usable - Data CI includes tools to manage, locate, access, manipulate, and analyze data, mechanisms to maintain confidentiality, and tools to facilitate creation and management of metadata- Data CI will involve strong, international, inter-agency and public-private partnerships
Challenges include: - Managing and analyzing very large datasets - Managing, analyzing, and using streaming data - Developing tools to permit research using confidential data
COVO and LWD: To appear (March)
Strategic Plans for Data, COVO and LWD
(FY 2006 – 2010)
Observatories - Based on ability to federate data-sets and data streams, some include instrument control, event detection and response, and some degree of virtualization - Examples: NVO, OOI, EarthScope, NEON, GEOSS
Virtual organizations - A geographically dispersed community with common interests that uses cyberinfrastructure to integrate a variety of digital resources into a common working environment
Technologies - Portals, workflows, data analysis, models, streaming data, event detection, instrument/observatory control, networking, authentication/authorization, digital libraries, …
The growth of observatories and virtual organizations
NSF Middleware Initiative (NMI)
• 20 awards totaling $9M– 10 “System Integrator” awards
• Focus – to further develop the integration and support infrastructure of middleware for the longer term
– 10 smaller awards focused on near-term capabilities and tool development
• Focus – to encourage the development of additional new middleware components and capabilities for the NMI program
Develop, deploy and sustain a set of reusable and expandable middleware functions that benefit many science and engineering applications in a networked environment.
Use "open standards“; foster national and international collaboration; sustainable; scaleable and securable
IRNC: 2005 Awards
• Awards– TransPAC2 (U.S. – Japan and beyond)– GLORIAD, (U.S. – China – Russia –
Korea)– Translight/PacificWave (U.S. – Australia)– TransLight/StarLight, (U.S. – Europe)– WHREN (U.S. – Latin America)
International Research Network Connections
CI-TEAM
• A Foundation-wide effort to foster CI training and workforce devel’t• Started FY05 ($2.5M) - focused on demonstration projects• Anticipated funding in FY06: $10M - small and large activities
FY05: - 70 projects (101 proposals) received -11 projects fundedBroadening participation in CI Alvarez (FIU) – CyberBridges Crasta (VA Tech) – Project-Centric Bioinformatics Fortson (Adler) – CI-Enabled 21st C. Astronomy Training for HS Science Teachers Fox (IU) – Bringing MSI Faculty into CI & e-Science Communities Gordon (OhSU) – Leveraging CI to Scale-up a Computational Science u/g Curriculum Panoff (Shodor) – Pathways to Cyberinfrastructure: CI through Computational Science Takai (SUNY Stonybrook) – High School Distributed Search for Cosmic Ray
Developing & implementing resources for CI workforce development
DiGiano (SRI) – Cybercollaboration between Scientists and Software Developers Figueiredo (U FL) – In-VIGO/Condor-G Middleware for Coastal and Estuarine CI Training Regli (Drexel) – CI for Creation and Use of Multi-disciplinary Engineering Models Simpson (PSU) – CI-based Engineering Repositories for Undergraduates (CIBER-U)
How it all fits together…
HPC
LWDCOVO
Teragrid (ETF)HPC acquisitions
CI-TEAMTeragrid (ETF)IRNCNMI
Teragrid (ETF)NMI
DATA
What would foster this integration?
What are the trends in campus cyberinfrastructure?
The integration of campus cyberinfrastructure and
national cyberinfrastructure
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Goal: to create and maintain a powerful, stable, persistent, and widely accessible cyberinfrastructure to enable the work of science and engineering researchers and educators across the nation.
NSF-Cyberinfrastructure