Computer Science 320

Parallel ComputingDesign Patterns

Problem Solving: How to Start?

• See if your problem fits into a class of problems that have already been solved

• Look for a suggestion for your solution in that class of solutions

Design Patterns

• A design pattern provides a template for suggested solutions to a class of siliarly structured problems

• Identify a design pattern that best matches your problem

Parallel Design Patterns

Three patterns in 1989 paper by Carriero and Gelernter:

– Result parallelism

– Agenda parallelism

– Specialist parallelism

Result Parallelism

Good for processing each element in a data structure, such as the pixels in an image or the frames in a movie

Ideally, the results of the computations are independent of each other

Result Parallelism

Bottleneck: sequential dependencies, where one result must await the computation of another

Positions of multiple stars in a time sequence

Spreadsheet recalculations

Result Parallelism with Dependencies

Agenda Parallelism

Good for computing one result from a large number of inputs

See if any DNA sequences match a query sequence

May also run into sequential dependencies, where tasks must wait

Agenda Parallelism: BLAST

Basic Local Alignment Search Tool

Unlike result parallelism, only interested in some results or combination thereof

Agenda Parallelism with Reduction

Compute in parallel and then apply a reduction operator

Specialist Parallelism

Each processor performs a specialized task on a series of data items (also known as pipelining)

Specialist Parallelism

For each star Calculate position Render image Store in PNG file

What if There Aren’t Enough Processors?

Large problems have billions of results to compute or tasks to perform, but we don’t yet have billions of processors

The specialist pattern usually requires fewer processors

Result Pattern: Clumping/Slicing

Clumping: lump many conceptual processors into one real processor

Slicing: partition a data structure into pieces and dedicate a process to each piece

Agenda Pattern: Clumping/Slicing

Clumping: lump many conceptual processors into one real processor

Slicing: partition a data structure into pieces and dedicate a process to each piece

Agenda Pattern: Master-Worker

A conceptual design usually for clusters

The master processor manages the agenda of tasks, and delegates these to the worker processors

The master receives the results and combines them

For Next Time

Introduction to parallel Java, and a first parallel program!