Highlights of Computationally-Based

Science at UAF Greg Newby, Director

Arctic Region Supercomputing Center University of Alaska Fairbanks

Internet2 Joint Techs Meeting July 11 2011, 8:37-8:55am

Abstract Computation modeling, computational analysis, data gathering and remote sensing are examples of computationally-based science at UAF. The campus benefits from a supercomputing center, campus computing facilities, departmental laboratories, and several specialized research units with strong computational components. This presentation will highlight several of these units and their current approaches to computationally-based research. In addition, these will be related to field research, instruction, outreach, and other aspects of the University's activities. Some of these research areas include northern climate and weather, oceanography, biological systems, space weather, ice (glaciers, sea ice, ice sheets, icing, and ice fog), network research, and northern languages and cultures.

Outline   A little about ARSC, and other cyberinfrastructure

in Alaska

  Snapshot of campus connectivity

  Focus on single sign-on technology, campus storage cloud, and related cross-UAF initiatives

  Selected highlights of computational science areas at UAF

  Conclusion & future direction

ARSC Basics   Supercomputing center, opened in 1993

  Campus + regional focus, funding from campus sources plus a variety of grants and contracts

  Until recently, ARSC was one of the DoD High Performance Computing Modernization Program centers. Since June 1, we are subcontractors in support of the HPCMP

  28 staff members (including 3 faculty), 4 students   Next 3 slides: ARSC cyberinfrastructure resources   For more info: www.arsc.edu

The “pacman” supercomputer   Login Nodes:

  2- Six core 2.2 GHz AMD Opteron Processors; 24 GB of memory per node. (2 GB per core); Mellanox Infiniband DDR Network Card

  9- Dual core 2.6 Ghz AMD Opteron processors, 65 GB memory

  88- Sixteen Core Compute Nodes:   2- Eight core 2.3 GHz AMD Opteron

Processors; 64 GB of memory per node (4 GB per core); QDR Infiniband Network Card; 250 GB local disk

  44- Twelve Core Compute Nodes:   2- Six core 2.2 GHz AMD Opteron

Processors; 32 GB of memory per node (2.6 GB per core); Mellanox Infiniband DDR Network Card

  4- Large Memory Nodes:   4- Eight core 2.3 GHz AMD Opteron

Processors; 256 GB of memory per node (8 GB per core); QDR Infiniband Network Card; 1 TB local disk

•  2- GPU Nodes: •  2- Quad core 2.4 GHz Intel CPUs; 64

GB of memory per node (8 GB per core); QDR Infiniband Network Card; Dual M2050 nVidia Fermi GPU cards

•  Mellanox QDR Infiniband Interconnect •  89 TB Panasas version 12 scratch file

system

Bigdipper ($ARCHIVE)   4- 8-core T2+ CoolThread processors (32

cores total); 1.4 GHz clock frequency

  SAM-QFS hierarchical storage managements system. Minimum 2 copies on tape; 1st copy on disk for most files

  Attached disk includes:   Sun Storage 6780 Array, with 125 TB of

general disk cache

  225 TB attached LSI storage (SATA-II)

  18 TB attached DDN storage (FC) used for caching small files

  5- FC8 dual port cards

  2- 10 gigabit Ethernet cards

  Numerous other connected technologies, providing high availability SAN

SL8500 Tape Library   Capable of storing over 30

PB (petabytes) of data with current generation tapes & drives (T10000C)

  Multiple independent Handbots™ travel on four rails (at up to 2.5 meters per second) for high performance data retrieval and redundancy

  An integrated service bay allows uninterrupted data access while typical maintenance tasks are performed

Connectivity to UAF   The basics:

  Distances are vast

  Population is small   Relatively few higher ed, research and high-tech

organizations   Costs are high!

  A history of using multiple networking technologies, following best of breed and maximum cost/benefit

Alaska Fiber Cables

As of October 2010, via UAF OIT 9

Prudhoe Bay

Fairbanks

To Nedonna Beach, OR

Whittier

Kenai

Anchorage

Homer

Seward

Valdez

Pedro BayIliamna

Port Alsworth

Nondalton

IgiugigLevelock

Glennallen

Juneau

Angoon

SitkaPetersburg

Wrangell

Ketchikan

Alaska’s Long-haul Subsea and Terrestrial Fiber

To Warrenton, ORTo Norma

Beach, WATo Florence,

OR

AU East

AU West

AKORNNorthstar

GCI AU WestGCI AU East

KKFL

GCI AU Northeast

GCI SEAFAST

GCI AU Northwest

ACS Fiber

GCI AU Northslope

AT&T Fiber

AU Northslope

ACS (RR Route)

AU Northwest(Parks Hwy Route)

AU Northeast(TAPS ROW Route)

TAPS ROW Route

AT&T (Richardson/Glenn Hwy Route)

Kodiak

Narrow Cape

GCI TERRA SW (2011)

Homer Electric Assoc.

Map: GCI

As of October 2010, via UAF OIT

•  Part of the Geophysical Institute at UAF

•  Satellite Tracking Ground Station

•  Synthetic Aperture Radar

•  GeoData Center

EPSCoR Cyberinfrastructure October 7-8, 2010 University of Alaska

11

Photo: ASF/UAF

Numerous other “big science, big data” facilities are part of UAF or part of UAF’s research infrastructure: Poker Flats Rocket Range, HAARP/MUIR, Toolik Lake LTER … many others!

Campus CI Initiatives   These are all undergoing rapid progress, and will be

deployed for testing during summer/fall 2011:   Single sign on

  Institutional repository   Data portal   Computational portal

  Campus storage cloud

  Major UAF partners implementing these include ARSC, OIT, Library, IARC, offices of VCR & Provost, and others

Single Sign On (SSO)   Impetus: Campus Office of Information

Technology (OIT) is a single holding location for information about persons (via the Registrar, Banner, Payroll, and other systems). Desire to avoid duplication of effort across different units on campus, and to have more seamless access to campus electronic resources

  Status: Very capable LDAP/Shibboleth deployment at www.edir.alaska.edu ; campus constituents meeting to further standardize services and expectations

Institutional Repository   Impetus: Need for better centralized systems for

tracking organizational data. Relies upon SSO

  Planning: Examining DSpace, first deployment will be July 2011

  Initial service: electronic archives for theses and dissertations (which are already cataloged by the Library, but not centralized online)

  Further service: faculty activity tracking such as publications

Data Portal   Impetus: Requirement for data management and archive

functionality for NSF/NIH/other grant proposals

  Status: Many extremely capable campus data providers exist; no desire to replace or combine these

  Goals:   Itemize campus data providers, and provide a single directory

to them at www.data.alaska.edu   Provide federated search across providers   Enable better access control and tracking, via SSO etc.   Provide value-added search results, such as combined

datasets or subsets; or automatically generating new datasets based on user queries

  Provide other modern tools for data use and reuse, including social media links, subscription services, etc.

Computational Portal   Impetus: better ease of use and shorter time to

solution, compared to command-line interfaces to computational tools; outreach and instruction for relatively new users

  Status: LSI/Bio portal is very sophisticated; others already exist (such as Tsunami Portal). These are mostly home-grown. Looking at HubZero for deployment of further portals

  Close integration with data portal, SSO, and campus cloud

Campus Storage Cloud   Impetus: need for transparent long-term storage for

broad campus constituencies; large-scale storage at ARSC is largely behind command-line and traditional (FTP/scp) access tools

  Goals: using SSO for campus constituencies, provide simple, transparent, scalable, secure storage for instructional, research, and organizational purposes

  Status: Exporting via NFS, starting SMB, to campus constituents; awaiting SSO for wider deployment

Some Computational Science Highlights

  These are just examples …

•  Visualization Research

•  ASSERT Center •  Remote Access Lab –

Available nation wide •  Digital Forensics Lab

•  SCADA Lab

•  GENI Site Participant

Bioinformatics Power Wall

UAF CS Department

•  GINA (Geographic Information Network of Alaska) receives and processes satellite data, and works with state, national and commercial mapping services for high quality elevation and land surface models

•  Remote sensing products provided to a wide constituency for research, policy making, and logistics

•  GINA relies on the ARSC/UA storage infrastructure for their 50TB+ data archive

20

gina.alaska.edu

•  PISM (Parallel Ice Sheet Model) is an internationally utilized model for ice sheets

•  Primary developer: UAF’s Prof. Ed Bueler

•  Used for IPCC modeling, to forecast ice sheet changes and scenarios for associated sea level rise

•  Driven by paleoclimate simulations; simulations span 125,000+ years

And many more…   Cyber-enabled and

computationally-based research at UAF is highly visible

  Often, these are in direct partnership with Arctic-focused field research

  Other areas include:   Native languages, cultures   Digital archives: images,

field recordings   Space physics

  Observational systems: meteorological stations, stream gauges, ocean buoys

  Genomic analysis, from sampling, to genotyping, to modeling; focus on Polar biomes

  Climate, weather: coupled ice, ocean, atmosphere, land surface

  Permafrost measurement and modeling, land surface change

  Molecular dynamics   Many, many others (e.g.,

over 70 ARSC computational “projects”)

North to the Future!   UAF has many areas of research emphasis, most of

which have a particular Arctic focus   There is a deep and growing involvement of

computationally-based techniques with theory, experiment, and observation

  ARSC and other campus units work together to provide diverse and capable computational and storage resources

  SSO and related technologies are cornerstones of integrated campus computational services for research, instruction, and outreach

Additional Factoids…   IPv6: ARSC has deployed extensively within

the .arsc.edu and .arsc.alaska.edu campus network LAN, and also end-to-end to arsc.edu and the DREN as part of our work with the US Dept. of Defense