Towards a Comprehensive Environment for Data Analysis

and VisualizationPamela Gillman, John Clyne

Users Forum May 19, 2005

Overview

• Introduction

• Mission

• Mission Goals

• Current Services

• Integrating Scientific Work Flow

• Centralized Data Access

• Contact Information

• Questions?

Introduction

• Data Analysis Services Group

• Formed in November 2004

• Pamela Gillman, John Clyne, Joey Mendoza

IMAGeInstitute for Mathematical

Applications in the Geosciences

DSSData

Support Section

ESSEnterprise

Services Section

SSGSupercomputingServices Group

DASGData Analysis

Services Group

MSSGMass Store

Services Group

CSGConsulting

Services Group

HSSHigh-End

Services Section

NETSNetworking Engineering & Telecommunication

Section

VETSVisualization & Enabling

Technologies Section

CSSComputational

Sciences Section

SCDScientific Computing Division

CISLComputational & Information

System Laboratory

Mission Statement

Enhancing scientific workflow by providing the research community with a highly advanced computing environment tailored toward the specialized needs of interactive data post-processing, analysis and visualization.

Mission Goals• Providing a state of the art, unified computing

environment comprised of powerful, closely coupled, interactive processing and visualization engines, high-performance storage, and specialized software applications

• Providing large-capacity, high-performance file systems shared between heterogeneous computers used for modeling, data analysis, visualization and access

• Evaluating, exploring and integrating applicable new technologies and methodologies to support the mission

• Producing visual content (animations and imagery) as a support service on behalf of scientific staff

• Performing basic research and algorithm development of relevant visualization and analysis methods

Current Services (1)

• Tempest– Interactive and batch data processing and analysis

– Model validation

– Applications• IDL, MATLAB, NCL, NCO

– Libraries• IMSL, NAG

– Development Environment• MIPSPro C, C++, F77, F90

• MPI, OMP

• GNU tools

• flint, WorkShop, WorkShopMPF

• Advanced visualization facilities– 3D and 4D data

– Terascale data sets

• Remote visualization services

• Applications and libraries:– Vtk, Vis5d, IDL, NCL,

OpenDX, and more

• Expert visualization staff

• Production visualization animation and imaging

• Customers:– Scientists

– Support staff (e.g. VETS)

Current Services (2)

Current Services (3)• SCD Visualization Lab

– Large format, high-resolution stereoscopic display

– Group meeting room

– Access Grid

– Operated by SCD’s Visualization and Enabling Technologies Section (VETS)

The SCD VisLab: A decision making environment for NCAR sciencesPhoto: Carlye Calvin -- UCAR

Reservations and info : http://www.vets.ucar.edu/Vislab/index.shtml

Video Switch

High Bandwidth Switch

SGI Onyx3800 Irix/MIPS Visualization SMP

SGI Origin2000 Irix/MIPS Analysis

SMPCiprico Shared Storage Array

Internet

Internet

64bit commodity visualization cluster

Visual Computing and Display Resources

SGI Prism Linux/Itanium2 Visualization SMP

Gfx pipe

Gfx pipe

Gfx pipe

Gfx pipe

GigE

FC

cpu 0

cpu 9

High bandwidth shared storage

Visual Supercomputer

Local visualization

Internet

images

X11 protocol

Remote visualization

Office Based Remote Visualization

Integrating Scientific Work Flow

Problem

Computation

AnalyzeResults

VisualizeResults

PublishResults

Centralized Data Access

high-performancefile system

Analysis Servers

VIS ServersCXFS

Compute Servers GPFS

MSS Servers cache

data portal SCD portal

SNFS

Contact Information

• SCD Customer Support– http://www.scd.ucar.edu/main.user.html– consult1@ucar.edu– 303-497-1278

• DASG– http://www.scd.ucar.edu/hss/dasg/– dasg@ucar.edu

Questions?