CS 838: Pervasive Parallelism

Introduction to OpenMPCopyright 2005 Mark D. HillUniversity of Wisconsin-Madison

Slides are derived from online references of Lawrence Livermore National Laboratory, National Energy Research Scientific Computing Center, University of Minnesota, OpenMP.orgThanks!

*(C) 2005

OutlineIntroductionMotivationExampleProgramming ModelExpressing parallelismSynchronizationSyntax

*(C) 2005

Introduction to OpenMPWhat is OpenMP?Open specification for Multi-ProcessingStandard API for defining multi-threaded shared-memory programsHeaderPreprocessor (compiler) directivesLibrary CallsEnvironment Variables

*(C) 2005

Motivation

Thread libraries are hard to useP-Threads/Solaris threads have many library calls for initialization, synchronization, thread creation, condition variables, etc.Usually require alternate programming stylesProgrammer must code with multiple threads in mind

Synchronization between threads introduces a new dimension of program correctness

*(C) 2005

Motivation

Wouldnt it be nice to write serial programs and somehow parallelize them automatically?

OpenMP can parallelize many serial programs with relatively few annotations that specify parallelism and independence

OpenMP is a small API that hides cumbersome threading calls with simpler directives

*(C) 2005

Example: Hello, World!

Files:(Makefile)hello.c

*(C) 2005

OutlineIntroductionMotivationExampleProgramming ModelExpressing parallelismSynchronizationSyntax

*(C) 2005

Programming ModelThread-like Fork/Join model

Typically multiple thread creation/ destruction events

Some built-in automatic parallelization

ForkJoin

*(C) 2005

Programming ModelsData parallelismThreads perform similar functions, guided by thread identifier

Control parallelismThreads perform differing functionsOne thread for I/O, one for computation, etc

*(C) 2005

Programming ModelMaster ThreadThread with ID=0Only thread that exists in sequential regionsDepending on implementation, may have special purpose inside parallel regionsSome special directives affect only the master thread (like master)ForkJoin

*(C) 2005

Programming ModelParallel Code Sections

Loop annotation directives can be used to parallelize loops

Explicit parallel regions are declared with the parallel directiveStart of region corresponds to N pthread_create() calls (Fork Event)End of region corresponds to N pthread_join() calls (Join Event)

*(C) 2005

Programming ModelSynchronization Constructs

Synchronization provided by OpenMP specificationCan declare Critical Sections in codeMutual exclusion guaranteed at runtimeCan declare simple statements as atomicBarrier directivesLock functions

*(C) 2005

Programming ModelDirectives

Directives (preprocessor) used to express parallelism and independence to OpenMP implementation

Some synchronization directivesAtomic, Critical Section, Barrier

*(C) 2005

Programming ModelsLibrary Calls

Library calls provide functionality that cannot be built-in at compile-time

Mutator/Accessor functionsomp_[get,set]_num_threads()

Lock/Unlock functionality

*(C) 2005

Programming ModelEnvironment Variables

Provide default behavior

Allow other processes to change behavior of OpenMP-enabled programs

Useful for scripting:setenv OMP_NUM_THREADS 4

*(C) 2005

LimitationsOpenMP isnt guaranteed to divide work optimally among threadsHighly sensitive to programming styleOverheads higher than traditional threadingRequires compiler support (use cc)Doesnt parallelize dependencies

*(C) 2005

OutlineIntroductionMotivationExampleProgramming ModelExpressing parallelismSynchronizationSyntax

*(C) 2005

OpenMP SyntaxGeneral syntax for OpenMP directives

Directive specifies type of OpenMP operationParallelizationSynchronizationEtc.Clauses (optional) modify semantics of Directive

#pragma omp directive [clause] CR

*(C) 2005

OpenMP SyntaxPARALLEL syntax

Ex: Output: (N=4)#pragma omp parallel{ printf(Hello!\n);} // implicit barrierHello!Hello!Hello!Hello!#pragma omp parallel [clause] CRstructured_block

*(C) 2005

OpenMP SyntaxDO/for Syntax (DO-Fortran, for-C)

Ex:#pragma omp parallel{ #pragma omp for private(i) shared(x) \ schedule(static,x/N) for(i=0;i