dr. rupak biswas – project manager 10 february 2011 · advertises rupak biswas, hecc project...

Dr. Rupak Biswas – Project Manager NASA Advanced Supercomputing (NAS) Division NASA Ames Research Center, Moffett Field, CA [email protected] (650) 604-4411

10 February 2011

•  Using the Pleiades supercomputer, aerospace engineers at NASA Langley have produced advanced simulations of the Orion Crew Exploration Vehicle (CEV) during reentry into the atmosphere, in support of thermal protection system (TPS) design for NASA’s next-generation space vehicle.

•  Over the past year, the resulting analyses have been used to: predict surface heating environments for TPS design; understand unsteady wake flow behind the vehicle; assess flow interactions and surface heating caused by the roll, yaw, and pitch jets used during descent; and reduce uncertainties associated with physical flow phenomenon coupled with chemistry when designing new spacecraft.

•  This work has required extensive HECC resources to compute the complex flow structures in the CEV; each unsteady calculation solves for over 200 million unknown quantities, and requires about 31,000 CPU- hours on Pleiades for each trajectory point analyzed.

•  HECC supercomputing resources have enabled unprecedented simulations that play a key role in understanding reentry environments to design a next-generation vehicle.

Mission Impact: By understanding the intense heating environment of a reentry spacecraft, the Agency can attain its spaceflight mission goals safely and efficiently."

POC: Stephen J. Alter, [email protected], (757) 864-7771, NASA Langley Research Center

Figure: Simulation of CEV wake flow structure showing asymmetry of the computed flowfield.

HECC Resources Support Analysis of CEV Reentry Flow Environments

10 February 2011 2 High End Computing Capabilities Project

•  HECC’s Application Performance and Productivity (APP) experts have improved the performance of a Satellite Data Simulation Unit (SDSU) program developed at NASA Goddard.

•  The code is used by NASA’s Global Modeling and Assimilation Office to analyze precipitation satellite data as part of a workflow to predict rainfall.

•  APP staff first analyzed the time-consuming routines using HECC-developed profiling tools to identify performance hot spots; data in a “do-loop” had to be reordered before OpenMP was used to improve the parallel efficiency of poorly performing regions of the code.

•  The parallel code scales almost linearly—achieving an 8-fold speedup using 8 OpenMP threads on a Nehalem node of Pleiades.

Mission Impact: By reducing the runtime of NASA Goddard’s Satellite Data Simulation code, users can now process multiple satellite datasets in a fixed window of time during a mission.

Performance Gains Attained on Satellite Data Simulation Unit (SDSU) Code

10 February 2011

POC: Samson Cheung, [email protected], (650) 604-0923, NASA Advanced Supercomputing Division, Computer Sciences Corp.

Figure: Precipitation observation with a high-density Tropical Rainfall Measuring Mission (TRMM) Microwave Imager (TMI) satellite dataset. The data captures the important feature in the center of the domain (where the tropical cyclone Erin was observed).

3 High End Computing Capabilities Project

10 February 2011

•  In late December, eight new “Westmere” racks were added to the Pleiades cluster, increasing the system’s peak performance to 1.081 petaflops.

•  The new racks add 6,144 Westmere cores and expands Pleiades to a total of 156 racks with 91,136 cores, which increases the computational capability of the system by 72 teraflops.

•  HECC and SGI staff successfully added the new racks via a unique live integration technique, which enabled the system to remain available to users while the racks were added.

•  This process saved over 3 million hours of computing time that would have been unavailable if the system had to be disabled for the integration work.

Mission Impact: The continued expansion of Pleiades provides increased computational capability to all NASA Mission Directorates.

POC: Bob Ciotti, [email protected], (650) 604-4408, NASA Advanced Supercomputing Division

Figure: Eight new SGI Westmere racks were installed on Pleiades, which added another 72 teraflops of computational capability to the system.

Installation of Additional Pleiades Westmere Racks Completed


High-End Computing Capabilities Project 5

•  The HECC Network team has developed Firewall Query Tool to help manage the highly complex access control environment on NASLAN, the NASA local area network. The benefits of the tool include: –  Greatly reduces the amount of time needed while

engineers troubleshoot to identify where a firewall problem is in the network, by performing complex, automated queries end-to-end.

–  Learns the dynamic routing table of each router on the network and updates firewall rules daily; this makes the software “network-aware” and able to calculate the path traffic takes through the network, as well as identify which firewalls are crossed.

–  Allows non-technical users to easily and instantly determine if there is an access restriction on the network, without the need to contact an engineer.

–  Identifies which specific firewall rules permit or block traffic, so engineers know exactly which rule to modify.

•  A whitepaper describing the capabilities of Firewall Query Tool can be found at: http://www.nas.nasa.gov/News/Techreports/2011/PDF/nas-11-001.pdf

Mission Impact: The Firewall Query Tool enables HECC support staff to quickly identify service network accessibility across multiple security domains, and helps identify weaknesses in HECC’s security defense.

POC: Nichole Boscia, [email protected], (650) 604-0891, NASA Advanced Supercomputing Division, Computer Sciences Corp.

Figure: The above diagram shows how different systems can traverse the network across various routes. Some are permitted by the firewall, while others are blocked when trying to reach a service.

Network Team Develops Firewall Query Tool to Manage Access Control


•  HECC users representing the NASA Advanced Supercomputing (NAS) Division at NASA Ames participated in the 49th AIAA Aerospace Sciences Meeting, held January 4–11, 2011, in Orlando.

•  Users shared and disseminated their latest knowledge and research results, obtained using HECC resources, with aerospace scientists and engineers from around the world.

•  Eight scientists presented papers and chaired/participated in panels on various applications of computational fluid dynamics to NASA aeronautics and aerospace projects; see slide #11.

•  In addition, HECC staff supported the NASA exhibit (with participation from seven other technical organizations) to explain the Agency’s featured work to conference attendees and to hand out printed materials.

•  The exhibit drew many visitors and was very well received; notable visitors to the booth were Administrator Charles Bolden, Chief Technologist Bobbie Braun, and industry partner executives from Boeing, General Electric, and Lockheed Martin.

Mission Impact: HECC support for NASA's AIAA exhibit enhanced the experience and interest of booth visitors, and helped increase the dissemination of NASA's state-of-the-art aerospace CFD research and engineering results.

POCs: Cetin Kiris, [email protected], (650) 604-4485, Gina Morello, [email protected], (650) 604-4462, NASA Advanced Supercomputing Division

Figure: Aerospace scientists and experts from around the world visited NASA’s AIAA booth in Orlando.

HECC Supports Technical Collaboration, Outreach at Annual Aerosciences Meeting

High-End Computing Capabilities Project 6 10 February 2011 6 High End Computing Capabilities Project

Presentations •  Vision, Experience, and Investigation Projects at the NAS Supercomputing Center, W. Thigpen,

Universidad de Guadalajara, Mexico, January 26, 2011. •  HECC Overview, W. Thigpen, Tecnológico de Monterrey in Guadalajara, Mexico, January 26, 2011. •  Supercomputing at NASA: Current Impact, Future Challenges, W. Thigpen, Keynote address at the

Paraninfo of the Universidad de Guadalajara, Mexico, January 27, 2011. Papers •  Testing an Eddy-Permitting Model of the Southern Ocean Carbon Cycle Against Observations, M.

Woloszyn, M. Mazloff, Dec. 2010, Elsevier, doi:10.1016/j.ocemod.2010.12.004.* •  Crash: A Block-Adaptive-Mesh Code for Radiative Shock Hydrodynamics -Implementation and

Verification, B. van der Holst, et al, arXiv:1101.758v1 [astro-ph.SR] January 19, 2011.* •  Learning from the Outer Heliosphere: Interplanetary Coronal Mass Ejection Sheath Flows and the

Ejecta Orientation in the Outer Atmosphere,” R.M. Evans, M. Opher, T.I. Gombosi, The Astrophysical Journal 728 (2011) 41, doi:10.1088/0004-637X/728/1/41.*

•  Firewall Policy Query Tool Whitepaper, N. Boscia, January 2011.

Presentations and Papers

* HECC provided supercomputing resources and services in support of this work


http://www.sciencedirect.com/science/article/pii/S1463500310001800

http://arxiv.org/abs/1101.3758

http://iopscience.iop.org/0004-637X/728/1/41/apj_728_1_41.text.html

http://www.nas.nasa.gov/News/Techreports/2011/2011.html

Papers (continued) •  AIAA Aerospace Science Meeting, January 11–14, 2011, Orlando *

  OVERFLOW Validation for Predicting Plume Impingement of Underexpanded Axisymmetric Jets onto Angled Flat Plates, H. Lee, G. Klopfer.

  Validation of OVERFLOW for Computing Plume Effects during the Ares 1 Stage Separation Process, J. Kless, H. Lee, G. Klopfer, J. Onufer, S. Pandya, W. Chan.

  Numerical Investigation of the Flow Angularity Effects of the NASA Langley UPWT on the Ares I DAC1 0.01-Scale Model, H. Lee, G. Klopfer

  Plume-Induced Flow Separation over a Cone-Cylinder Flare Body, J. Kless, G. Klopfer.   Best Practices for Aero-Database CFD Simulations of Ares V Ascent, C. Kiris, J. Housman, M.

Gusman.   Best Practices for CFD Simulations of Launch Vehicle Ascent with Plumes, M. Gusman, J.

Housman, C. Kiris.   Dynamic Error Estimation and Mesh Refinement in Aerodynamic Shape Design, M. Nemec, M.

Aftosmis.

Presentations and Papers



News and Events

10 February 2011

•  Penn State CI Day to Showcase Cyberinfrastructure, Gant Daily, January 16, 2011 – advertises Rupak Biswas, HECC Project Manager, as presenter.

•  ISUM Supercomputing Conference HECC Deputy Project Manager Bill Thigpen participated in the Instituto Universitario de Mercadotechnia (ISUM) Conference, hosted by the University of Guadalajara, Mexico; Thigpen’s visit promoted the use of supercomputing among research communities; numerous online pieces announced this visit.

•  California Academy of Sciences: ‘Life: A Cosmic Story’ San Francisco Chronicle, January 21, 2011 – The HECC Data Analysis and Visualization group contributed significantly to this project and the article mentions NASA Ames



NAS Utilization

10 February 2011

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Pleiades Columbia Production

Share Limit

Job Drain

Dedtime Drain

Limits Exceeded

Specific CPUs

Insufficient CPUs

Held

Queue Not Schedulable

Not Schedulable

No Jobs

Dedicated

Down

Degraded

Boot

Used

January 2011


NAS Utilization Normalized to 30-Day Month

10 February 2011

0

10,000,000

20,000,000

30,000,000

40,000,000

50,000,000

60,000,000

70,000,000

80,000,000

Stan

dard

Bill

ing

Uni

ts

NAS

NLCS

NESC

SOMD

SMD

ESMD

ARMD

Alloc. to Orgs


NAS Utilization Normalized to 30-Day Month

10 February 2011

1 Allocation to orgs. increased to 80% 2 SMD augmentation 3 RTJones retired 4 32 Westmere racks added 5 Schirra retired, 4 Westmere racks added 6 RTJones compensation removed 7 8 Westmere racks added


Tape Archive Status

10 February 2011

0

2

4

6

8

10

12

14

16

18

20

22

24

26

28

30

32

34

36

Unique File Data Total Tape Data Tape Capacity Tape Library Capacity

Peta

Byt

es

Capacity Used HECC Pre-mission NAS NLCS NESC SOMD SMD ESMD ARMD

January 2011

The data for Total Tape Data and Unique Tape Data include three tape libraries. We are in the process of migrating data and have run into hardware problems, so there is currently duplicate data. That is why the usage appears to exceed capacity.


Tape Archive Status

10 February 2011

0

2

4

6

8

10

12

14

16

18

20

22

24

26

28

30

32

34

36

Dec-09 Jan-10 Feb-10 Mar-10 Apr-10 May-10 Jun-10 Jul-10 Aug-10 Sep-10 Oct-10 Nov-10 Dec-10 Jan-11

Peta

Byt

es Tape Library Capacity

Tape Capacity Total Tape Data Unique Tape Data Unique File Data

The data for Total Tape Data and Unique Tape Data include three tape libraries. We are in the process of migrating data and have run into hardware problems, so there is currently duplicate data. That is why the usage appears to exceed capacity.


Pleiades: SBUs Reported, Normalized to 30-Day Month

10 February 2011

0

10,000,000

20,000,000

30,000,000

40,000,000

50,000,000

60,000,000

70,000,000

80,000,000

Stan

dard

Bill

ing

Uni

ts (h

ours

* .9

[Har

pert

own/

Wes

tmer

e] o

r 1.1

[Neh

alem

])

NAS

NLCS

NESC

SOMD

SMD

ESMD

ARMD

Alloc. to Orgs


Pleiades: Monthly SBUs by Run Time

10 February 2011

0

2,000,000

4,000,000

6,000,000

8,000,000

10,000,000

12,000,000

14,000,000

16,000,000

0 - 1 hours > 1 - 4 hours > 4 - 8 hours > 8 - 24 hours > 24 - 48 hours

> 48 - 72 hours

> 72 - 96 hours

> 96 - 120 hours

> 120 hours Stan

dard

Bill

ing

Uni

ts (h

ours

* .9

[Har

pert

own/

Wes

tmer

e] o

r 1.1

[Neh

alem

])

Job Run Time (hours) January 2011


Pleiades: Monthly Utilization by Size and Mission

10 February 2011

0

2,000,000

4,000,000

6,000,000

8,000,000

10,000,000

12,000,000

14,000,000

16,000,000

18,000,000

20,000,000

1 - 32 33 - 64 65 - 128 129 - 256 257 - 512 513 - 1024 1025 - 2048

2049 - 4096

4097 - 8192

8193 - 16384

16385 - 32768 St

anda

rd B

illin

g U

nits

(hou

rs *

.9 [H

arpe

rtow

n/W

estm

ere]

or 1

.1 [N

ehal

em])

Job Size (cores)

NAS NLCS NESC SOMD SMD ESMD ARMD

January 2011

0 100,000 200,000 300,000 400,000 500,000 600,000 700,000 800,000

8193 - 16384

16385 - 32768


Pleiades: Monthly Utilization by Size and Length

10 February 2011

0

2,000,000

4,000,000

6,000,000

8,000,000

10,000,000

12,000,000

14,000,000

16,000,000

18,000,000

20,000,000

1 - 32 33 - 64 65 - 128 129 - 256 257 - 512 513 - 1024

1025 - 2048

2049 - 4096

4097 - 8192

8193 - 16384

16385 - 32768 St

anda

rd B

illin

g U

nits

(hou

rs *

.9 [H

arpe

rtow

n/W

estm

ere]

or 1

.1 [N

ehal

em])

Job Size (cores)

> 120 hours > 96 - 120 hours > 72 - 96 hours > 48 - 72 hours > 24 - 48 hours > 8 - 24 hours > 4 - 8 hours > 1 - 4 hours 0 - 1 hours

January 2011

0 100,000 200,000 300,000 400,000 500,000 600,000 700,000 800,000

8193 - 16384

16385 - 32768


Pleiades: Average Time to Clear All Jobs

10 February 2011

0

24

48

72

96

120

144

168

192

216

240

264

288

312

336

Feb-10 Mar-10 Apr-10 May-10 Jun-10 Jul-10 Aug-10 Sep-10 Oct-10 Nov-10 Dec-10 Jan-11

Hou

rs

ARMD ESMD SMD SOMD/NESC


Pleiades: Average Expansion Factor

10 February 2011

1.00

1.50

2.00

2.50

3.00

3.50

4.00

4.50


ARMD ESMD SMD SOMD NESC

4.90 7.89


Columbia: SBUs Reported, Normalized to 30-Day Month

10 February 2011

0

1,000,000

2,000,000

3,000,000

4,000,000

5,000,000

6,000,000

7,000,000

8,000,000

9,000,000

10,000,000

Stan

dard

Bill

ing

Uni

ts (h

ours

* 1)

NAS

NLCS

NESC

SOMD

SMD

ESMD

ARMD

Alloc. to Orgs


Columbia: Monthly SBUs by Run Time

10 February 2011

0

50,000

100,000

150,000

200,000

250,000

300,000

350,000

400,000

450,000

500,000

0 - 1 hours > 1 - 4 hours > 4 - 8 hours > 8 - 24 hours > 24 - 48 hours

> 48 - 72 hours

> 72 - 96 hours

> 96 - 120 hours

> 120 hours

Stan

dard

Bill

ing

Uni

ts (h

ours

* 1)

Job Run Time (hours) January 2011


Columbia: Monthly Utilization by Size and Mission

10 February 2011

0

100,000

200,000

300,000

400,000

500,000

600,000

700,000

800,000

900,000

1 - 32 33 - 64 65 - 128 129 - 256 257 - 512 513 - 1024 1025 - 2048

Stan

dard

Bill

ing

Uni

ts (h

ours

* 1)

Job Size (cores)

NAS NLCS NESC SOMD SMD ESMD ARMD

January 2011

0

2,000

4,000

6,000

8,000

10,000

12,000

513 - 1024 1025 - 2048


Columbia: Monthly Utilization by Size and Length

10 February 2011

0

100,000

200,000

300,000

400,000

500,000

600,000

700,000

800,000

900,000

1 - 32 33 - 64 65 - 128 129 - 256 257 - 512 513 - 1024 1025 - 2048

Stan

dard

Bill

ing

Uni

ts (h

ours

* 1)

Job Size (cores)

> 120 hours > 96 - 120 hours > 72 - 96 hours > 48 - 72 hours > 24 - 48 hours > 8 - 24 hours > 4 - 8 hours > 1 - 4 hours 0 - 1 hours

January 2011

0

2,000

4,000

6,000

8,000

10,000

12,000

513 - 1024 1025 - 2048


Columbia: Average Time to Clear All Jobs

10 February 2011

0

24

48

72

96

120

144

168

192


Hou

rs

ARMD ESMD SMD SOMD/NESC

465


Columbia: Average Expansion Factor

10 February 2011

1.00

1.50

2.00

2.50

3.00

3.50

4.00

4.50


ARMD ESMD SMD SOMD NESC

24.80 8.84


dr. rupak biswas – project manager 10 february 2011 · advertises rupak biswas, hecc project...

Documents