Part 3

What are the general types of parallelism that we already discussed?

What are the general types of parallelism that we already discussed?

1. instruction-level

2. processor-level

Types:1. Array2. Vector3. Multiprocessor/multicore4. Multicomputer5. COWs (cluster of workstations)

Tanenbaum, Structured Computer Organization, Fifth Edition, (c)

2006 Pearson Education, Inc. All rights reserved. 0-13-148521-0

Type 1: array processor- ex. ILLIAC IV- 1 CPU; 64 AUs (arithmetic units)

Type 2: vector processor 1 CPU w/ (a small number, say 8) vector registers

but (typically) only one AU

example is Intel MMX, SSE1 – SSE4 SIMD instuctions SSE: 8 128-bit registers (xmm0..xmm7)

Each 128-bit register may contain: 4 32-bit fp numbers 2 64-bit fp numbers 4 32-bit integers 8 16-bit ints 16 8-bit ints

Tanenbaum, Structured Computer Organization, Fifth Edition, (c)

2006 Pearson Education, Inc. All rights reserved. 0-13-148521-0

Type 3: multiprocessor multicomputer with optional local memories and shared memory; common bus typically 64 or less CPUs OpenMP (Multi-Processing; openmp.org)

Type 4: multicomputer

like multiprocessor, but no shared memory

as many as 10,000 CPUs

different topologies (such as 2d grid, 3d grid, trees, or rings) to speed communications

Type 5: COW (Cluster Of Workstations)

COTS (Common Off The Shelf) systems communicating via COTS network hardware

MPI standard programming library Message Passing Interface http://www-unix.mcs.anl.gov/mpi/ (not to be confused with OpenMP)