1

SIAC 2000 Program

2

SIAC 2000 at a Glance

AM Lunch PM Dinner

Sun Condor

Mon NOW HPC Globus Clusters

Tue PVMMPI Clusters HPVM

Wed Condor HPVM

3

SIAC 2000 Materials: Book 1

• David Spector, “Building Linux Clusters: Scaling Linux for Scientific and Enterprise Applications,” August 2000

• CD-ROM RedHat Linux, PVM, …• O'Reilly & Associates, ISBN:

1565926250

4

SIAC 2000 Materials: Book 2

• Rajkumar Buyya (Editor), “High Performance Cluster Computing: Programming and Applications” Volume 2, June 1999

• Prentice Hall; ISBN: 0130137855

5

SIAC 2000 Materials: Book 3

• Selection of papers– Linux clusters– Beowulf HOWTO– PVM, MPI

• FAQs• Handouts from speakers

6

Introduction toIntroduction toCluster ComputingCluster Computing

Prabhaker MatetiWright State University

Dayton, Ohio

7

Overview

• High performance computing• High throughput computing• NOW, HPC, and HTC • Parallel algorithms• Software technologies

8

“High Performance” Computing

• CPU clock frequency• Parallel computers• Alternate technologies

– Optical– Bio– Molecular

9

“Parallel” Computing

• Traditional supercomputers– SIMD, MIMD, pipelines– Tightly coupled shared memory– Bus level connections– Expensive to buy and to maintain

• Cooperating networks of computers

10

“NOW” Computing

• Workstation• Network• Operating System• Cooperation• Distributed (Application) Programs

11

Traditional Supercomputers

• Very high starting cost– Expensive hardware– Expensive software

• High maintenance• Expensive to upgrade

12

Traditional Supercomputers

No one is predicting their demise, but …

13

Computational Grids

are the future

14

Computational Grids

“Grids are persistent environments that enable software applications to integrate instruments, displays,

computational and information resources that are managed by

diverse organizations in widespread locations.”

15

Computational Grids

• Individual nodes can be supercomputers, or NOW

• High availability• Accommodate peak usage• LAN : Internet :: NOW : Grid

16

Globus: A Computational Grid

• Lee Liming, Argonne

• Monday, Aug 21, 2000, 1:00 – 5:30 PM

17

“NOW” Computing

• Workstation• Network• Operating System• Cooperation• Distributed+Parallel Programs

18

Workstation? PC? Mac?

19

“Workstation Operating System”

• Authenticated users• Protection of resources• Multiple processes• Preemptive scheduling• Virtual Memory• Hierarchical file systems• Network centric

20

Network

• Ethernet– 10 Mbps obsolete– 100 Mbps common– 1000 Mbps desired

• Protocols– TCP/IP

21

Cooperation

• Workstations are “personal”• Others use slows you down• …• Willing to share• Willing to trust

22

Distributed Programs

• Spatially distributed programs– A part here, a part there, …– Parallel– Synergy

• Temporally distributed programs– Compute half today, half tomorrow– Combine the results at the end

• Migratory programs– Have computation, will travel

23

Technological Bases of Distributed+Parallel Programs

• Spatially distributed programs– Message passing

• Temporally distributed programs– Shared memory

• Migratory programs– Serialization of data and programs

24

Distributed Shared Memory

• “Simultaneous” read/write access by spatially distributed processors

• Abstraction layer of an implementation built from message passing primitives

• Semantics not so clean

25

Technological Bases for Migratory programs

• Same CPU architecture– X86, PowerPC, MIPS, SPARC, …, JVM

• Same OS + environment• Be able to “checkpoint”

– suspend, and – then resume computation – without loss of progress

26

Development of Distributed+Parallel Programs

• New code + algorithms• Old programs rewritten in new

languages that have distributed and parallel primitives

• Parallelize legacy code

27

New Programming Languages

• With distributed and parallel primitives

• Functional languages• Logic languages• Data flow languages

28

Parallel Programming Languages

• based on the shared-memory model

• based on the distributed-memory model

• parallel object-oriented languages • parallel functional programming

languages• concurrent logic languages

29

PVM, and MPI

• Message passing primitives• Can be embedded in many existing

programming languages• Architecturally portable• Open-sourced implementations

30

PVM, and MPI

• Prabhaker Mateti, WSU

• TuesdayAug 22, 20008:30 – 12:00

31

OpenMP

• Distributed shared memory API• Implementations: Real soon now

• http://www.openmp.org/

32

SPMD

• Single program, multiple data• Contrast with SIMD• Same program runs on multiple

nodes• May or may not be lock-step• Nodes may be of different speeds• Barrier synchronization

33

Condor

• Cooperating workstations• Migratory programs

– Checkpointing– Remote IO

• Resource matching

34

Condor

• Prabhaker Mateti, WSU

• WednesdayAug 22, 20008:30 – 12:00

35

Clusters of Workstations

• Inexpensive alternative to traditional supercomputers

• High availability– Lower down time– Easier access

• Development platform with production runs on traditional supercomputers

36

Linux Clusters

• Kumaran Kalyanasundaram, SGI

• Monday Aug 21, 20006:30 – 9:00 PM

• Tuesday Aug 22, 20001:00 – 5:30 PM

37

HPVM Clusters

• Mario Lauria

• WednesdayAug 23, 20001:00 – 4:00 PM