cs 838: pervasive parallelism introduction to openmp copyright 2005 mark d. hill university of...

of 31 /31
CS 838: Pervasive Parallelism Introduction to OpenMP Copyright 2005 Mark D. Hill University of Wisconsin-Madison Slides are derived from online references of Lawrence Livermore National Laboratory, National Energy Research Scientific Computing Center, University of Minnesota, OpenMP.org Thanks!

Author: hester-white

Post on 02-Jan-2016

212 views

Category:

Documents


0 download

Embed Size (px)

TRANSCRIPT

  • CS 838: Pervasive Parallelism

    Introduction to OpenMPCopyright 2005 Mark D. HillUniversity of Wisconsin-Madison

    Slides are derived from online references of Lawrence Livermore National Laboratory, National Energy Research Scientific Computing Center, University of Minnesota, OpenMP.orgThanks!

    *(C) 2005

    OutlineIntroductionMotivationExampleProgramming ModelExpressing parallelismSynchronizationSyntax

    *(C) 2005

    Introduction to OpenMPWhat is OpenMP?Open specification for Multi-ProcessingStandard API for defining multi-threaded shared-memory programsHeaderPreprocessor (compiler) directivesLibrary CallsEnvironment Variables

    *(C) 2005

    Motivation

    Thread libraries are hard to useP-Threads/Solaris threads have many library calls for initialization, synchronization, thread creation, condition variables, etc.Usually require alternate programming stylesProgrammer must code with multiple threads in mind

    Synchronization between threads introduces a new dimension of program correctness

    *(C) 2005

    Motivation

    Wouldnt it be nice to write serial programs and somehow parallelize them automatically?

    OpenMP can parallelize many serial programs with relatively few annotations that specify parallelism and independence

    OpenMP is a small API that hides cumbersome threading calls with simpler directives

    *(C) 2005

    Example: Hello, World!

    Files:(Makefile)hello.c

    *(C) 2005

    OutlineIntroductionMotivationExampleProgramming ModelExpressing parallelismSynchronizationSyntax

    *(C) 2005

    Programming ModelThread-like Fork/Join model

    Typically multiple thread creation/ destruction events

    Some built-in automatic parallelization

    ForkJoin

    *(C) 2005

    Programming ModelsData parallelismThreads perform similar functions, guided by thread identifier

    Control parallelismThreads perform differing functionsOne thread for I/O, one for computation, etc

    *(C) 2005

    Programming ModelMaster ThreadThread with ID=0Only thread that exists in sequential regionsDepending on implementation, may have special purpose inside parallel regionsSome special directives affect only the master thread (like master)ForkJoin

    *(C) 2005

    Programming ModelParallel Code Sections

    Loop annotation directives can be used to parallelize loops

    Explicit parallel regions are declared with the parallel directiveStart of region corresponds to N pthread_create() calls (Fork Event)End of region corresponds to N pthread_join() calls (Join Event)

    *(C) 2005

    Programming ModelSynchronization Constructs

    Synchronization provided by OpenMP specificationCan declare Critical Sections in codeMutual exclusion guaranteed at runtimeCan declare simple statements as atomicBarrier directivesLock functions

    *(C) 2005

    Programming ModelDirectives

    Directives (preprocessor) used to express parallelism and independence to OpenMP implementation

    Some synchronization directivesAtomic, Critical Section, Barrier

    *(C) 2005

    Programming ModelsLibrary Calls

    Library calls provide functionality that cannot be built-in at compile-time

    Mutator/Accessor functionsomp_[get,set]_num_threads()

    Lock/Unlock functionality

    *(C) 2005

    Programming ModelEnvironment Variables

    Provide default behavior

    Allow other processes to change behavior of OpenMP-enabled programs

    Useful for scripting:setenv OMP_NUM_THREADS 4

    *(C) 2005

    LimitationsOpenMP isnt guaranteed to divide work optimally among threadsHighly sensitive to programming styleOverheads higher than traditional threadingRequires compiler support (use cc)Doesnt parallelize dependencies

    *(C) 2005

    OutlineIntroductionMotivationExampleProgramming ModelExpressing parallelismSynchronizationSyntax

    *(C) 2005

    OpenMP SyntaxGeneral syntax for OpenMP directives

    Directive specifies type of OpenMP operationParallelizationSynchronizationEtc.Clauses (optional) modify semantics of Directive

    #pragma omp directive [clause] CR

    *(C) 2005

    OpenMP SyntaxPARALLEL syntax

    Ex: Output: (N=4)#pragma omp parallel{ printf(Hello!\n);} // implicit barrierHello!Hello!Hello!Hello!#pragma omp parallel [clause] CRstructured_block

    *(C) 2005

    OpenMP SyntaxDO/for Syntax (DO-Fortran, for-C)

    Ex:#pragma omp parallel{ #pragma omp for private(i) shared(x) \ schedule(static,x/N) for(i=0;i