shared-memory model and threads intel software college introduction to parallel programming – part...

Shared-Memory Model and Threads

Intel Software College

Introduction to Parallel Programming – Part 2

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.

2Shared-Memory Model and Threads

Intel® Software College

Objectives

At the end of this module you should be able to:

Describe the shared-memory model of parallel programming

Describe the differences between the fork/join model and the general threads model

Demonstrate how to implement domain and functional decompositions using threads

Decide whether a variable in a multithreaded program should be shared or private





Cooperating Parallel Processes

Parallel computing multiple processes working together to speed the solution of a task

Working together process cooperation

Kinds of cooperation

Sharing information (communication)

Keeping out of each other’s way (synchronization)





The Shared-Memory Model

Shared Memory

CPU

PrivateMemory

CPU

PrivateMemory

CPU

PrivateMemory





Evaluating Parallel Models

How do processes share information?

How do processes synchronize?

In shared-memory model, both accomplished through shared variables

Communication: buffer

Synchronization: semaphore





Methodology

Study problem, sequential program, or code segment

Look for opportunities for parallelism

Use threads to express parallelism





What Is a Process?

A program in some state of execution

Code

Data

Logical address space

Information about a process includes

Process state

Program counter

Values of CPU registers

Memory management information





What Is a Thread?

“A unit of control within a process” — Carver and Tai

Main thread executes program’s “main” function

Main thread may create other threads to execute other functions

Threads have own program counter, copy of CPU registers, and stack of activation records

Threads share process’s data, code, address space, and other resources

Threads have lower overhead than processes





Another Way of Looking at It

So why does this view matter?

Thread 0(Program Counter, Registers, etc.)

AddressSpace

File Descriptors

Other Data

Process





Why This View Matters


AddressSpace

File Descriptors

Other Data

Process






What Are Threads Good For?

Making programs easier to understand

Overlapping computation and I/O

Improving responsiveness of GUIs

Improving performance through parallel execution





Fork/Join Programming Model

When program begins execution, only master thread active

Master thread executes sequential portions of program

For parallel portions of program, master thread forks (creates or awakens) additional threads

At join (end of parallel section of code), extra threads are suspended or die





MasterThread

Fork JoinJoin Fork

Oth

er

Th

read

s





Relating Fork/Join to Code

for {

}

for {

}

Sequential code

Parallel code

Sequential code

Parallel code

Sequential code





More General Threads Model

When program begins execution, only one user thread, called the main thread, is active

The main thread can create other threads, which execute other functions

Created threads can also create additional threads

How this is done varies according to programming language or API





Fork/Join versus General Model

You can implement fork/join in the general model

Hence fork/join a special case of general model

More structured

More easily optimized

General model

More flexible

Better support for functional decompositions





Incremental Parallelization

Sequential program a special case of threaded program

Programmers can add parallelism incrementally

Profile program execution

Repeat

Choose best opportunity for parallelization

Transform sequential code into parallel code

Until further improvements not worth the effort





Utility of Threads

Threads are flexible enough to implement

Domain decomposition

Functional decomposition

Pipelining





Domain Decomposition Using Threads

SharedMemory

Thread 0 Thread 2

Thread 1

f ( )

f ( )

f ( )





SharedMemory

Functional Decomposition Using Threads

Thread 0 Thread 1

e ( )

g ( )h ( )

f ( )





Pipelining Using Threads

Thread 0 Thread 2Thread 1

Shared Memory

Input Output

e ( ) f ( ) g ( )

Dataset 2

Data sets5, 6, ...

Dataset 4 Data

set 3 Dataset 1





Shared versus Private Variables

SharedVariables

PrivateVariables

PrivateVariables

Thread

Thread





Domain Decomposition

Sequential Code:

int a[1000], i;

for (i = 0; i < 1000; i++) a[i] = foo(i);






Sequential Code:

int a[1000], i;

for (i = 0; i < 1000; i++) a[i] = foo(i);

Thread 0:

for (i = 0; i < 500; i++) a[i] = foo(i);

Thread 1:

for (i = 500; i < 1000; i++) a[i] = foo(i);






Sequential Code:

int a[1000], i;

for (i = 0; i < 1000; i++) a[i] = foo(i);

Thread 0:

for (i = 0; i < 500; i++) a[i] = foo(i);

Thread 1:

for (i = 500; i < 1000; i++) a[i] = foo(i);

SharedPrivate





Functional Decompositionint e;

main () { int x[10], j, k, m; j = f(x, k); m = g(x, k); ...}

int f(int *x, int k){ int a; a = e * x[k] * x[k]; return a;}

int g(int *x, int k){ int a; k = k-1; a = e / x[k]; return a;}





Functional Decompositionint e;

main () { int x[10], j, k, m; j = f(x, k); m = g(x, k);}



Thread 0

Thread 1





Functional Decompositionvolatile int e;




Thread 0

Thread 1

Static variable: Shared









Thread 0

Thread 1

Heap variable: Shared






main () { int x[10], j, k, m; j = f(x, k); m = g(x. k);}



Thread 0

Thread 1

Function’s local variables: Private





Shared and Private Variables

Shared variables

Static variables

Heap variables

Contents of run-time stack at time of call

Private variables

Loop index variables

Run-time stack of functions invoked by thread





The Shared-Memory Model (Reprise)

Shared Memory

CPU

PrivateMemory

CPU

PrivateMemory

CPU

PrivateMemory





The Threads Model

Shared Variables

Thread

PrivateVariables

Thread

PrivateVariables

Thread

PrivateVariables





References

Jim Beveridge and Robert Wiener, Multithreading Applications in Win32®, Addison-Wesley (1997).

David R. Butenhof, Programming with POSIX® Threads, Addison-Wesley, (1997).

Richard H. Carver and Kuo-Chung Tai, Modern Multithreading: Implementing, Testing, and Debugging Java and C++/Pthreads/ Win32 Programs, Wiley-Interscience (2006).

Doug Lea, “A Java Fork/Join Framework,” Proceedings of the ACM 2000 Conference on Java Grande, pp. 36-43.

Michael J. Quinn, Parallel Programming in C with MPI and OpenMP, McGraw-Hill (2004).

shared-memory model and threads intel software college introduction to parallel programming – part...

Documents

intel logo

general threads model

forkjoin model

united states

respective owners

private slide

threads objectives

shared variables communication