1

In Summary

Need more computing power Improve the operating speed of processors

& other components constrained by the speed of light,

thermodynamic laws, & the high financial costs for processor fabrication

Connect multiple processors together & coordinate their computational efforts

parallel computers allow the sharing of a computational task

among multiple processors

2

Technology Trends...

Performance of PC/Workstations components has almost reached performance of those used in supercomputers… Microprocessors (50% to 100% per year) Networks (Gigabit SANs); Operating Systems (Linux,...); Programming environment (MPI,…); Applications (.edu, .com, .org, .net, .shop, .bank);

The rate of performance improvements of commodity systems is much rapid compared to specialized systems.

3

Technology Trends

4

Trend

[Traditional Usage] Workstations with Unix for science & industry vs PC-based machines for administrative work & work processing

[Trend] A rapid convergence in processor performance and kernel-level functionality of Unix workstations and PC-based machines

5

Rise and Fall of Computer Architectures

Vector Computers (VC) - proprietary system: provided the breakthrough needed for the emergence of

computational science, buy they were only a partial answer.

Massively Parallel Processors (MPP) -proprietary systems:

high cost and a low performance/price ratio. Symmetric Multiprocessors (SMP):

suffers from scalability Distributed Systems:

difficult to use and hard to extract parallel performance. Clusters - gaining popularity:

High Performance Computing - Commodity Supercomputing

High Availability Computing - Mission Critical Applications

6

The Dead Supercomputer Societyhttp://www.paralogos.com/DeadSup

er/ ACRI Alliant American

Supercomputer Ametek Applied Dynamics Astronautics BBN CDC Convex Cray Computer Cray Research

(SGI?Tera) Culler-Harris Culler Scientific Cydrome

Convex C4600

Dana/Ardent/Stellar Elxsi ETA Systems Evans & Sutherland

Computer Division Floating Point Systems Galaxy YH-1 Goodyear Aerospace

MPP Gould NPL Guiltech Intel Scientific

Computers Intl. Parallel Machines KSR MasPar

Meiko Myrias Thinking

Machines Saxpy Scientific

Computer Systems (SCS)

Soviet Supercomputers

Suprenum

7

Computer Food Chain: Causing the demise of specialize systems

•Demise of mainframes, supercomputers, & MPPs

8The promise of supercomputing to the average PC User ?

Towards Clusters

9

Towards Commodity Parallel Computing

linking together two or more computers to jointly solve computational problems

since the early 1990s, an increasing trend to move away from expensive and specialized proprietary parallel supercomputers towards clusters of workstations

Hard to find money to buy expensive systems the rapid improvement in the availability of commodity

high performance components for workstations and networks

Low-cost commodity supercomputing from specialized traditional supercomputing platforms

to cheaper, general purpose systems consisting of loosely coupled components built up from single or multiprocessor PCs or workstations

10

History: Clustering of Computers for Collective

Computing

1960 1990 1995+1980s 2000+

PDAClusters

11

Why PC/WS Clustering Now ?

Individual PCs/workstations are becoming increasing powerful

Commodity networks bandwidth is increasing and latency is decreasing

PC/Workstation clusters are easier to integrate into existing networks

Typical low user utilization of PCs/WSs Development tools for PCs/WS are more mature PC/WS clusters are a cheap and readily available Clusters can be easily grown

12

What is Cluster ?

A cluster is a type of parallel or distributed processing system, which consists of a collection of interconnected stand-alone computers cooperatively working together as a single, integrated computing resource.

A node a single or multiprocessor system with memory, I/O

facilities, & OS generally 2 or more computers (nodes) connected

together in a single cabinet, or physically separated & connected

via a LAN appear as a single system to users and applications provide a cost-effective way to gain features and benefits

13

Cluster Architecture

Sequential Applications

Parallel Applications

Parallel Programming Environment

Cluster Middleware

(Single System Image and Availability Infrastructure)

Cluster Interconnection Network/Switch

PC/Workstation

Network Interface Hardware

Communications

Software

PC/Workstation

Network Interface Hardware

Communications

Software

PC/Workstation

Network Interface Hardware

Communications

Software

PC/Workstation

Network Interface Hardware

Communications

Software

Sequential Applications

Sequential Applications

Parallel ApplicationsParallel

Applications

14

So What’s So Different about Clusters?

Commodity Parts? Communications Packaging? Incremental Scalability? Independent Failure? Intelligent Network Interfaces? Complete System on every node

virtual memory scheduler files …

Nodes can be used individually or jointly...

15

Windows of Opportunities

Parallel Processing Use multiple processors to build MPP/DSM-like systems for

parallel computing Network RAM

Use memory associated with each workstation as aggregate DRAM cache

Software RAID Redundant array of inexpensive disks Use the arrays of workstation disks to provide cheap, highly

available, & scalable file storage Possible to provide parallel I/O support to applications Use arrays of workstation disks to provide cheap, highly

available, and scalable file storage Multipath Communication

Use multiple networks for parallel data transfer between nodes

16

• Enhanced Performance (performance @ low cost)

• Enhanced Availability (failure management)

• Single System Image (look-and-feel of one system)

• Size Scalability (physical & application)

• Fast Communication (networks & protocols)

• Load Balancing (CPU, Net, Memory, Disk)

• Security and Encryption (clusters of clusters)

• Distributed Environment (Social issues)

• Manageability (admin. And control)

• Programmability (simple API if required)

• Applicability (cluster-aware and non-aware app.)

Cluster Design Issues

17

Summary: Cluster Advantage

Price/performance ratio is low when compared with a dedicated parallel supercomputer.

Incremental growth that often matches with the demand patterns.

The provision of a multipurpose system Scientific, commercial, Internet applications

Have become mainstream enterprise computing systems: In 2003 List of Top 500 Supercomputers, over

50% of them are based on clusters and many of them are deployed in industries.