fspec20 bars

8/9/2019 Fspec20 Bars

1/124

OpenMP Fortran Application ProgramInterface

Version 2.0, November 2000

1

2

3

Line Numbers Added: Sat Oct 28 09:45:59 CDT 2000


2/124


3/124

Contents

Page

Introduction [1] 1

S cope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

G lossa ry . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

E xecut ion Mod el . . . . . . . . . . . . . . . . . . . . . . . . . 3

C omplia n ce . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Or ga n iza t ion . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Directives [2] 7Open MP D ir ect ive F orm a t . . . . . . . . . . . . . . . . . . . . . . 7

D irect ive S en t in els . . . . . . . . . . . . . . . . . . . . . . . . 8

F ixed S ource F or m D ir ect ive S en t in els . . . . . . . . . . . . . . . . . 8

F r ee S our ce F or m D irect ive S en t in el . . . . . . . . . . . . . . . . . 8

C om ment s I nside D ir ect ives . . . . . . . . . . . . . . . . . . . . . 10

C ommen t s in D ir ect ives w it h F ixed S ource F or m . . . . . . . . . . . . . . 10

C ommen t s in D ir ect ives w it h F ree S our ce F orm . . . . . . . . . . . . . . 10

C on dit ion a l C ompila t ion . . . . . . . . . . . . . . . . . . . . . . 10

F ixed S ource F or m C on dit ion a l C ompila tion S en t in els . . . . . . . . . . . . 11

F r ee S our ce F or m C on dit iona l C ompila tion S en t in el . . . . . . . . . . . . . 11

P a r a llel Region C on st r uct . . . . . . . . . . . . . . . . . . . . . . . 12

Wor k-sh a rin g C on st r uct s . . . . . . . . . . . . . . . . . . . . . . . 15

DO D ir ect ive . . . . . . . . . . . . . . . . . . . . . . . . . . 15

SECTIONS D irect ive . . . . . . . . . . . . . . . . . . . . . . . . 18

SINGLE D ir ect ive . . . . . . . . . . . . . . . . . . . . . . . . 20

WORKSHARE D irect ive . . . . . . . . . . . . . . . . . . . . . . . 20

C ombin ed P a ra llel Work-sh a r in g C onst r uct s . . . . . . . . . . . . . . . . . 22

PARALLEL DO D ir ect ive . . . . . . . . . . . . . . . . . . . . . . 23PARALLEL SECTIONS D irect ive . . . . . . . . . . . . . . . . . . . . 24

PARALLEL WORKSHARE D ir ect ive . . . . . . . . . . . . . . . . . . . 24

Synchronization Constructs and the MASTER D irect ive . . . . . . . . . . . . . . 25

MASTER D ir ect ive . . . . . . . . . . . . . . . . . . . . . . . . 25

Version 2.0, November 2000 i

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34


4/124

Contents OpenMP Fortran Application Program Interface

Page

CRITICAL D irect ive . . . . . . . . . . . . . . . . . . . . . . . . 26

BARRIER D ir ect ive . . . . . . . . . . . . . . . . . . . . . . . . 26

ATOMIC D ir ect ive . . . . . . . . . . . . . . . . . . . . . . . . 27

FLUSH D irect ive . . . . . . . . . . . . . . . . . . . . . . . . . 29

ORDERED D ir ect ive . . . . . . . . . . . . . . . . . . . . . . . . 30

D a t a E n vir on ment C on st r uct s . . . . . . . . . . . . . . . . . . . . . 31

THREADPRIVATE D irect ive . . . . . . . . . . . . . . . . . . . . . . 32

D a t a S cope At t r ibut e C la uses . . . . . . . . . . . . . . . . . . . . 34

PRIVATE C la use . . . . . . . . . . . . . . . . . . . . . . . . 35

SHARED C la use . . . . . . . . . . . . . . . . . . . . . . . . 36

DEFAULT C la use . . . . . . . . . . . . . . . . . . . . . . . . 36

FIRSTPRIVATE C la use . . . . . . . . . . . . . . . . . . . . . . 37

LASTPRIVATE C la use . . . . . . . . . . . . . . . . . . . . . . 38

REDUCTION C la use . . . . . . . . . . . . . . . . . . . . . . . 38

COPYIN C la use . . . . . . . . . . . . . . . . . . . . . . . . 41

COPYPRIVATE C la use . . . . . . . . . . . . . . . . . . . . . . 41

D a t a E nvir on men t Rules . . . . . . . . . . . . . . . . . . . . . . 42

D ir ect ive B ind in g . . . . . . . . . . . . . . . . . . . . . . . . . 45

D ir ect ive Nest ing . . . . . . . . . . . . . . . . . . . . . . . . . 45

Run-time Library Routines [3] 47

E xecut ion E nvir on men t Rout ines . . . . . . . . . . . . . . . . . . . . 47OMP_SET_NUM_THREADS S ubr out in e . . . . . . . . . . . . . . . . . . 48

OMP_GET_NUM_THREADS F un ct ion . . . . . . . . . . . . . . . . . . . 48

OMP_GET_MAX_THREADS F un ct ion . . . . . . . . . . . . . . . . . . . 49

OMP_GET_THREAD_NUM F unct ion . . . . . . . . . . . . . . . . . . . . 49

OMP_GET_NUM_PROCS F un ct ion . . . . . . . . . . . . . . . . . . . . 50

OMP_IN_PARALLEL F un ct ion . . . . . . . . . . . . . . . . . . . . . 50

OMP_SET_DYNAMIC S ubrout in e . . . . . . . . . . . . . . . . . . . . 51

OMP_GET_DYNAMIC F un ct ion . . . . . . . . . . . . . . . . . . . . . 51

OMP_SET_NESTED S ubrout in e . . . . . . . . . . . . . . . . . . . . 52

OMP_GET_NESTED F unct ion . . . . . . . . . . . . . . . . . . . . . 52

L ock Rout in es . . . . . . . . . . . . . . . . . . . . . . . . . . 52

OMP_INIT_LOCK a n d OMP_INIT_NEST_LOCK S ubrout ines . . . . . . . . . . . . 54

OMP_DESTROY_LOCK a n d OMP_DESTROY_NEST_LOCK S ubr out in es . . . . . . . . . 54

ii Version 2.0, November

2000

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69


5/124

OpenMP Fortran Application Program Interface Contents

Page

OMP_SET_LOCK a n d OMP_SET_NEST_LOCK S ubr out in es . . . . . . . . . . . . 54

OMP_UNSET_LOCK a nd OMP_UNSET_NEST_LOCK S ubr out in es . . . . . . . . . . . 55

OMP_TEST_LOCK a n d OMP_TEST_NEST_LOCK F unct ion s . . . . . . . . . . . . 55Timin g Rout ines . . . . . . . . . . . . . . . . . . . . . . . . . 56

OMP_GET_WTIME F unct ion . . . . . . . . . . . . . . . . . . . . . . 56

OMP_GET_WTICK F unct ion . . . . . . . . . . . . . . . . . . . . . . 57

Environment Variables [4] 59

OMP_SCHEDULE E n vir on m en t Va r ia ble . . . . . . . . . . . . . . . . . . . 59

OMP_NUM_THREADS E nvir on men t Va r ia ble . . . . . . . . . . . . . . . . . 60

OMP_DYNAMIC E n viron men t Va r ia ble . . . . . . . . . . . . . . . . . . . 60

OMP_NESTED E n viron men t Va r ia ble . . . . . . . . . . . . . . . . . . . 61

Appendix A Examples 63

E xecut in g a S imple L oop in P a r a llel . . . . . . . . . . . . . . . . . . . 63

S pecify in g C ond it ion a l C om pila t ion . . . . . . . . . . . . . . . . . . . 63

U sin g P a r a llel Region s . . . . . . . . . . . . . . . . . . . . . . . 64

U s in g t h e NOWAIT C la use . . . . . . . . . . . . . . . . . . . . . . . 64

U s in g t h e CRITICAL D irect ive . . . . . . . . . . . . . . . . . . . . . 64

U s in g t h e LASTPRIVATE C la use . . . . . . . . . . . . . . . . . . . . . 65

U s in g t h e REDUCTION C la use . . . . . . . . . . . . . . . . . . . . . 65

S pecify in g P a r a llel S ect ion s . . . . . . . . . . . . . . . . . . . . . . 67

U s in g SINGLE D ir ect ives . . . . . . . . . . . . . . . . . . . . . . . 67

S pecify in g S eq uen t ia l Order in g . . . . . . . . . . . . . . . . . . . . . 68

S pecify in g a F ixed Number of Th rea ds . . . . . . . . . . . . . . . . . . 68

U s in g t h e ATOMIC D ir ect ive . . . . . . . . . . . . . . . . . . . . . . 69

U s in g t h e FLUSH D ir ect ive . . . . . . . . . . . . . . . . . . . . . . 69

D et er min in g t h e Num ber of Thr ea ds U sed . . . . . . . . . . . . . . . . . 70

U sin g L ocks . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

U sin g Nest a ble L ocks . . . . . . . . . . . . . . . . . . . . . . . . 71

Nested DO D ir ect ives . . . . . . . . . . . . . . . . . . . . . . . . 73

E xa mples S how in g I ncorr ect Nest in g of Wor k-sha rin g D ir ect ives . . . . . . . . . . 74

Binding of BARRIER D ir ect ives . . . . . . . . . . . . . . . . . . . . . 77

Scoping Varia bles w ith t he PRIVATE C la use . . . . . . . . . . . . . . . . . 78

Version 2.0, November 2000 iii

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102


6/124

Contents OpenMP Fortran Application Program Interface

Page

E xa m ples of Non complia n t S t ora ge Associa t ion . . . . . . . . . . . . . . . . 78

Ex a mp les o f Sy n t a x o f Pa r a l lel DO L oops . . . . . . . . . . . . . . . . . . 81

Examples of the ATOMIC D ir ect ive . . . . . . . . . . . . . . . . . . . . 82

Examples of the ORDERED D ir ect ive . . . . . . . . . . . . . . . . . . . 83

Examples of THREADPRIVATE D a t a . . . . . . . . . . . . . . . . . . . . 84

Examples of the Data Attribute Clauses: SHARED a n d PRIVATE . . . . . . . . . . . 88

Examples of the Data Attribute Clause: COPYPRIVATE . . . . . . . . . . . . . 89

Examples of the WORKSHARE D irect ive . . . . . . . . . . . . . . . . . . . 91

Appendix B Stubs for Run-time Library Routines 95

Appendix C Using the SCHEDULE Clause 101

Appendix D Interface Declaration Module 105

Example of an Interface Declaration INCLUDE F ile . . . . . . . . . . . . . . . 105

Example of a Fortran 90 Interface Declaration MODULE . . . . . . . . . . . . . 107

E xa mple of a G ener ic I n t er fa ce for a L ibra ry Rout in e . . . . . . . . . . . . . . 111

Appendix E Implementation-Dependent Behaviors in OpenMP Fortran 113

Appendix F New Features in OpenMP Fortran version 2.0 115

Tables

Ta ble 1. SCHEDULE C la use Va lues . . . . . . . . . . . . . . . . . . . 17

Ta ble 2. Reduct ion Va ria ble I nit ia liza t ion Va lues . . . . . . . . . . . . . . . 40

iv Version 2.0, November

2000

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122


7/124

Copyright © 1997-2000 OpenMP Architecture Review Board. Permission to copy

without fee all or part of this material is granted, prov ided the OpenMP Architecture

Rev iew B oard copy right n otice and the ti t le of this document a ppea r. Notice is giv en

tha t copy ing is b y permission of OpenMP Architecture Rev iew B oa rd.

123

124

125

126


8/124


9/124

Introduction [1]

This document specifies a collection of compiler directives, library routines, andenv ironment v ariab les that can b e used to specify shared memory parallelism in

Fortran programs. The functionality described in this document is collectively known

a s t h e OpenM P Fortran Appl icat ion Program Interface (API) . The goal of this

specifi cation is to provide a model for pa ra llel progra mming tha t is porta b le across

sha red memory ar chitectures from different v endors. The OpenMP Fortra n API is

supported by compilers from numerous vendors. More information about OpenMP

can b e found at the following web site:

http://www.openmp.org

The directiv es, l ib rary routines, a nd env ironment v aria b les defi ned in this document

will al low users to create and manage parallel programs while ensuring portab ili ty .The directiv es extend the F ortr an seq uential programm ing model with

single-progra m multiple dat a (S P MD ) constructs, work -sha ring constru cts a nd

sy nchronization constructs, and prov ide support for the sharing and priv atization of

data. The l ib rary routines and env ironment v ariab les prov ide the functionality to

control the run-time execution environment. The directive sentinels are structured so

t h a t t h e d i r ect i v es a r e t r e a t e d a s F o r t r a n com m e n t s . C o m pi le r s t h a t s u pp or t t h e

O pe n MP F or t r a n A P I i n cl u d e a c om m a n d l i n e op t i on t h a t a c t i v a t e s a n d a l low s

interpretation of all OpenMP compiler directives.

1.1 Scope

This specification describes only user-directed parallelization, wherein the user

explicitly specifi es th e a ctions t o b e ta k en b y t he compiler a nd r un-time sy st em in

order to execute the program in parallel . OpenMP Fortran implementations are not

required to check for dependencies, conflicts, deadlocks, race conditions, or other

problems that result in incorrect program execution. The user is responsible for

ensuring that the application using the OpenMP Fortran API constructs executes

correctly.

C ompiler-genera ted a utoma tic par a lleliza tion is not a ddressed in t his specifi cat ion.

1.2 Glossary

The following terms a re used in this document:

Version 2.0, November 2000 1

127

128

129

130

131

132

133

134

135

136

137

138139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157


10/124

Introduction [1] OpenMP Fortran Application Program Interface

d efin ed - For th e contents of a da ta ob ject, the property of ha v ing or b eing giv en a

v alid v alue. For the allocation status or association status of a data ob ject, the

property of hav ing or b eing giv en a v alid status.

do-construct - The Fortran S tandard term for the construct that specifi es the repeated

execution of a seq uence of executa b le stat ements. The Fortra n S ta nda rd calls such a

repeated sequence a loop . The loop that follows a DO or PARALLEL DO directive

cannot b e a WHILE loop or a DO loop w ithout loop cont rol.

implementation-dependent - A behav ior or v alue t ha t is implementa tion-dependent is

permitted to vary among different OpenMP-compliant implementations (possibly in

response t o l imitat ions of ha rdw a re or opera ting sy stem). Implementa tion-dependent

items a re l isted in Appendix E, page 113, a nd OpenMP -complia nt implementat ions

are req uired to document how these i tems are handled.

l exical extent - S t a tements lexica lly contained w ithin a structured b lock.

master thread - Th e t h r e a d t h a t cr e a t e s a t e a m w h e n a p a r a l le l r eg i on i s e n t er e d .

nested - a parallel region is said to b e nested i f i t appears within the dy namic extent

of a PARALLEL constru ct tha t (1) does not ha v e an IF clause or (2) ha s a n IF cla use

and the logical expression within the clause ev aluates to . TRUE..

noncom pliant - C ode structures or a rra ngements describ ed a s noncomplia nt a re not

required to be supported by OpenMP-compliant implementations. Upon encountering

such structures, an OpenMP-compliant implementation may produce a compiler error.

Ev en if a n implementa tion produces a n executa b le for a progra m containing such

structures, i ts execution ma y t erminat e premat urely or ha v e unpredictab le b eha v ior.

parallel region - S t a t e m en t s t h a t b i n d t o a n O pe n MP PARALLEL con s t r u ct a n d a r ea v ailab le for execution b y multiple threads.

p r i v a t e - Accessible to only one threa d in the team for a para llel region. Note tha t

t h e r e a r e s e ve r a l w a y s t o s p eci fy t h a t a v a r i a b l e is p r i va t e : u s e a s a l oca l v a r i a b l e in

a sub program called from a parallel region, in a THREADPRIVATE directive, in a

PRIVATE, FIRSTPRIVATE, LASTPRIVATE, or REDUCTION clause, or u se of the v a riab le

a s a loop contr ol v a riab le.

serialize – When a para llel region is serialized, i t is executed b y a single thr ead. A

para llel region is said to b e serialized i f an d only i f a t lea st one of t he following a re

true:

1. The logica l expression in a n IF clause attached to the parallel directiv e ev aluatest o .FALSE. .

2. I t is a nested pa ra llel region a nd nested para llelism is disab led.

3. I t is a nested para llel region an d t he implementa tion chooses to seria lize nested

parallel regions.

2 Version 2.0, November

2000

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180181

182

183

184

185

186

187

188

189

190191

192

193

194


11/124

OpenMP Fortran Application Program Interface Introduction [1]

serial region - S t a t e m en t s t h a t d o n ot b i n d t o a n O p e n MP PARALLEL construct. In

ot h e r w or d s , t h e s e s t a t e m e n t s a r e e xe cu t e d b y t h e m a s t e r t h r e a d ou t s i d e of a p a r a l le l

region.

shared - Accessib le to all t hreads in t he tea m for a para llel region.

structured block - A str uctured block is a collection of one or more execut a ble

s t a t e m en t s w i t h a s i n gl e poi n t o f en t r y a t t h e t o p a n d a s i n gl e poi n t o f ex it a t t h e

b ottom. Execution must alway s proceed with entry at the top of the b lock and exit at

the b ottom with only one exception: i t is al lowed to hav e a STOP s t a t e m en t i n s id e a

structured b lock . This statement has the well defi ned b ehav ior of terminating the

entire program.

undefined - For the contents of a da ta object, t he property of not ha v ing a determinat e

v alue. The result of a reference to a d a ta ob ject w ith undefi n ed contents is

unspecifi ed. For the allocation status or association status of a data ob ject, the

property of not ha v ing a v alid sta tus. The b eha v ior of an operat ion w hich relies uponan undefi ned allocation status or association status is unspecifi ed.

unspecified - A b eha v ior or result tha t is unspecifi ed is not constra ined b y

req uirements in the OpenMP Fortra n API. P ossib ly resulting from the misuse of a

langua ge constru ct or other error, such a b eha v ior or result ma y not b e k nowa b le prior

to the execution of a program, and may lead to premature termination of the program.

v a r i a b l e – A d a t a ob je ct w h o s e v a l u e ca n b e d efi n e d a n d r e de fi n e d d u r i n g t h e

e xe cu t i on o f a p r og r a m . I t m a y b e a n a m e d d a t a ob je ct , a n a r r a y e le m en t , a n a r r a y

section, a structure component, or a sub string.

white space - A seq uence of space or ta b cha ra cters.

1.3 Execution Model

The OpenMP Fortran API uses the fork-join model of parallel execution. A program

that is written with the OpenMP Fortran API b egins execution as a single process,

called the master thread of execution. The ma ster t hread executes seq uentia lly until

t h e fi r s t p a r a l l el c on s t r u ct i s e n cou n t e r ed . I n t h e O p en M P F o rt r a n A P I , t h e

PARALLEL/END PARALLEL directive pair constitut es the para llel constru ct. When a

parallel construct is encountered, the master thread creates a team o f t h r ea d s , a n d

t h e m a s t e r t h r e a d b ecom e s t h e m a s t e r o f t h e t e a m . Th e s t a t e m en t s i n t h e p r og r a m

that are enclosed b y the parallel construct, including routines called from within the

enclosed statements, are executed in parallel b y each thread in the team. The

statements enclosed lexically within a construct defi ne the lexical extent of the

construct. The d y n a m i c extent furt her includes t he routines called from w ithin t he

construct.


195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230


12/124

Introduction [1] OpenMP Fortran Application Program Interface

Upon completion of the parallel construct, the threads in the team sy nchronize and

only th e ma ster t hread continues execution. Any numb er of pa ra llel constructs can b e

specifi ed in a single program. As a result , a program may fork and join many times

during execution.

The OpenMP Fortra n AP I a llows progra mmers to use directiv es in r outines called

from within parallel constructs. D irectiv es that do not appear in the lexical extent of

the parallel construct b ut l ie in the dy namic extent are called orphaned d irectives.

Orphan ed directiv es a llow users to execute m a jor portions of their progra m in pa ra llel

wit h only minimal changes to th e seq uentia l progra m. With t his functiona lity, users

can code parallel constructs at the top lev els of the program call tree and use

directives to control execution in any of the called routines.

1.4 Compliance

An implementation of the OpenMP Fortran API is OpenM P-compli ant if it recognizes

and preserv es the semantics of al l the elements of this specifi cation as laid out in

chapters 1, 2, 3, a nd 4. The a ppendixes a re for informa tion purposes only a nd a re not

part of t he specifi cation.

The OpenMP Fortran API is an extension to the b ase language that is supported b y

an implementation. I f the b ase language does not support a language construct or

extension th a t a ppea rs in t his document, t he OpenMP implementa tion is not r eq uired

to support i t .

Al l s t a n d a r d F or t r a n i n t r i n s i cs a n d l i b r a r y r o u t in e s a n d F o r t r a n 90 ALLOCATE a n d

DEALLOCATE s tatements must b e thread- safe in a compliant implementation.

Unsy nchronized use of such intrinsics and routines b y dif ferent threads in a parallel

region must produce correct results ( though not n ecessarily th e sa me a s serial

execution results, as in the case of random number generation intrinsics, for example).

U n s y n c h r on i ze d u s e of F o r t r a n ou t p u t s t a t e m en t s t o t h e s a m e u n i t m a y r e s u lt i n

output in which data written b y dif ferent threads is interleav ed. S imilarly ,

u n s y n ch r on i z ed i n p u t s t a t e m en t s f r om t h e s a m e u n it m a y r e a d d a t a i n a n i n t e r l ea v e d

fashion. U nsy nchronized use of Fortra n I /O, such t ha t each th read a ccesses a

dif ferent unit , produces the sa me results a s serial execution of th e I/O st a tements.

I n b o t h F o r t r a n 90 a n d F or t r a n 9 5, a v a r i a b l e t h a t h a s e xp li ci t i n i t i a l iz a t i o nimplicitly has the SAVE at tr ib ute. This is not t he case in F ORTRAN 77. Howev er, an

implementation of OpenMP Fortran must giv e such a v ariab le the SAVE a t t r i b u t e,

regardless of th e v ersion of Fortra n u pon w hich i t is b ased.

The OpenMP Fortran API specifi es that certain b ehav ior is

“implementation-dependent”. A conforming OpenMP implementation is required to


2000

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261262

263

264

265

266


13/124

OpenMP Fortran Application Program Interface Introduction [1]

defi ne a nd document i ts b eha v ior in these cases. S ee Appendix E, pa ge 113, for a l ist

of implementation-dependent behaviors.

1.5 Organization

The rest of t his document is organized int o the following chapt ers:

• C ha pter 2, pa ge 7, describ es the compiler directiv es.

• C ha pter 3, pa ge 47, describ es the run-time lib rary routines.

• C ha pter 4, page 59, describ es the env ironment va riab les.

• Appendix A, pa ge 63, contains exa mples.

• Appendix B , page 95, describ es stub run-time lib rar y routines.

• Ap pe n d ix C , pa g e 101, h a s i n f or m a t i on a b o ut u s i n g t h e SCHEDULE cla use.

• Appendix D , pa ge 105, has exam ples of interfaces for the run-time libra ry routines.

• Appendix E, page 113, describ es implementa tion-dependent b eha v iors.

• Appendix F, page 115, describ es the new fea tures in the OpenMP Fortra n v 2.0 AP I.


267

268

269

270

271

272

273

274

275

276

277

278

279


14/124


15/124

Directives [2]

D i r ect i v es a r e s p eci a l F o r t r a n com m e n t s t h a t a r e i d en t i fi e d w i t h a u n i q u e sentinel .The directiv e sentinels are structured so that the directiv es are treated as Fortran

comments. C ompilers that support the OpenMP Fortran API include a command line

option that activ ates and allows interpretation of al l OpenMP compiler directiv es. In

the remainder of this document, the phrase OpenM P com pi lat i on i s u s e d t o m e a n

tha t OpenMP directiv es are int erpreted during compila tion.

This chapter addresses the following topics:

• S ection 2.1, page 7, describ es the directive forma t.

• S ection 2.2, page 12, describ es the para llel region construct.

• S ection 2.3, page 15, describ es the work -sha ring constructs.

• S ection 2.4, page 22, describ es the comb ined pa ra llel work -sha ring constru cts.

• S ection 2.5, page 25, describ es the sy nchroniza tion constructs a nd th e MASTER

directive.

• S ection 2.6, page 31, describ es the da ta env ironment, which includes directiv es

a n d c la u s e s t h a t a f f ect t h e d a t a e n vi r on m e n t .

• S ection 2.7, page 45, describ es directive b inding.

• S ection 2.8, page 45, describ es directive nesting.

2.1 OpenMP Directive Format

The forma t of an OpenMP directiv e is a s follows:

sent in el di r ecti ve_nam e [clause [[,] clause ]...]

All OpenMP compiler directives must begin with a directive sentinel . D irectiv es are

case- insensitiv e. C lauses can appear in any order af ter the directiv e name. C lauses

on directiv es can b e repeat ed a s needed, sub ject t o the restrictions l isted in t hedescription of each clause. Directives cannot be embedded within continued

statements, and statements cannot b e emb edded within directiv es. C omments

preceded b y an exclamation point may appear on the same line as a directiv e.

The following sections describe the OpenMP directive format:


280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307


16/124

Directives [2] OpenMP Fortran Application Program Interface

• S ection 2.1.1, page 8, describ es directiv e sentinels.

• Section 2.1.2, pa ge 10, describes comments inside directives.

• Section 2.1.3, pag e 10, describes conditional compila tion.

2.1.1 Directive Sentinels

The directive sentinels accepted by an OpenMP-compliant compiler differ depending

on t he Fortra n source form b eing used. The !$OMP sentinel is a ccepted wh en

compiling either fixed source form files or free source form files. The C$OMP a n d

*$OMP sentinels a re a ccepted only wh en compiling fi xed source form fi les.

The following sections contain more information on using the different sentinels.

2.1.1.1 Fixed Source Form Directive Sentinels

The OpenMP Fortra n AP I a ccepts the following sentinels in fi xed source form fi les:

!$OMP | C$OMP | *$OMP

S e n t i n el s m u s t s t a r t i n c ol u m n o n e a n d a p p ea r a s a s i n gl e w or d w i t h n o i n t e r v en i n g

wh ite space (spaces a nd/or t a b chara cters). Fortra n fi xed form line length , case

sensitiv ity, w hite spa ce, continua tion, a nd column rules a pply to t he directiv e l ine.

Initia l directiv e l ines must ha v e a space or zero in column six, a nd continua tion

directiv e l ines must hav e a character other than a space or a zero in column six.

Exa mple: The following forma ts for specify ing directiv es ar e eq uiv alent (the fi rst l ine

represents the position of the first 9 columns):

C23456789

!$OMP PARALLEL DO SHARED(A,B,C)

C$OMP PARALLEL DO

C$OMP+SHARED(A,B,C)

C$OMP PARALLELDOSHARED(A,B,C)

2.1.1.2 Free Source Form Directive Sentinel

The OpenMP Fortra n AP I a ccepts the following sentinel in free source form fi les:


2000

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333


17/124

OpenMP Fortran Application Program Interface Directives [2]

!$OMP

The sentinel can appear in any column as long as i t is preceded only b y white space

(s p a ce s a n d t a b c h a r a ct e r s ). I t m u s t a p p ea r a s a s i n gl e w or d w i t h n o i n t e r v en i n g

white space. Fortran free form line length, case sensitiv ity , white space, and

continua tion rules apply to the directiv e l ine. Initia l directiv e l ines must ha v e a space

a f t e r t h e s en t i n el . C on t i n u e d d ir e ct i v e li n es m u s t h a v e a n a m p e r s a n d a s t h e l a s t

nonb la nk cha ra cter on t he l ine, prior to a ny comment placed inside th e directiv e.

C ontinuation directiv e l ines can hav e an ampersand after the directiv e sentinel with

op t i on a l w h i t e s p a ce b e for e a n d a f t e r t h e a m p er s a n d .

One or more b lank s or tab s must b e used to separate adjacent k ey words in directiv es

in free source form, except in the following cases, where white space is optional

b etw een the giv en pair of k ey words:

END CRITICAL

END DO

END MASTER

END ORDERED

END PARALLEL

END SECTIONS

END SINGLE

END WORKSHARE

PARALLEL DO

PARALLEL SECTIONS

PARALLEL WORKSHARE

Exa mple: The following forma ts for specify ing directiv es a re eq uiv alent (the fi rst l ine

represents the position of the first 9 columns):

!23456789

!$OMP PARALLEL DO &

!$OMP SHARED(A,B,C)

!$OMP PARALLEL &

!$OMP&DO SHARED(A,B,C)

!$OMP PARALLELDO SHARED(A,B,C)

In order to simplify the presentation, the remainder of this document uses the !$OMP

sentinel.


334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366


18/124


2.1.2 Comments Inside Directives

The OpenMP Fortran API accepts comments placed inside directives. The rules

gov erning such comments depend on t he F ortra n source form b eing used.

2.1.2.1 Comments in Directives with Fixed Source Form

C omments may appear on the same line as a directiv e. The exclamation point

initiat es a comment wh en it a ppears a f ter column 6. The comment extends t o the end

of the source l ine and is ignored. I f the fi rst nonb lank character af ter the directiv e

sentinel of a n initial or continuat ion d irective l ine is a n excla ma tion point, t he l ine is

ignored.

2.1.2.2 Comments in Directives with Free Source Form C omments may appear on the same line as a directiv e. The exclamation point

initiat es a comment. The comment extends t o the end of th e source l ine a nd is

ignored. I f the fi rst nonb lank character af ter the directiv e sentinel is an exclamation

point, the line is ignored.

2.1.3 Conditional Compilation

The OpenMP Fortran API permits Fortran lines to be compiled conditionally. The

directiv e sentinels for conditiona l compila tion th a t a re a ccepted b y a n

OpenMP-compliant compiler depend on the Fortran source form being used. The !$sentinel is accepted when compiling either fixed source form files or free source form

files. The C$ a n d *$ sentinels are accepted only when compiling fixed source form.

D uring OpenMP compila tion, the sentinel is repla ced b y tw o spaces, a nd the rest of

t h e l in e i s t r e a t e d a s a n or m a l F o r t r a n l in e .

If an OpenMP-compliant compiler supports a macro preprocessor (for example, cpp),

the Fortran processor must defi ne the sy mb ol _OPENMP to be used for conditional

compila tion. This sy m b ol is defi ned during OpenMP compilation to h a v e the decimal

v a l u e YYYYMM w h e r e YYYY a n d MM are the y ear and month designations of the v ersion

of t h e O pe n MP F or t r a n AP I t h a t t h e i m p le m en t a t i on s u pp or t s .

The following sections contain more information on using the different sentinels for

conditional compilation. (See Section A.2, page 63, for an example.)


2000

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395


19/124


2.1.3.1 Fixed Source Form Conditional Compilation Sentinels

The OpenMP Fortran API accepts the following conditional compilation sentinels in

fi xed source form fi les:

! $ | C $ | * $ | c $

The sentinel must start in column 1 and appear as a single word with no interv ening

white space. Fortran fi xed form line length, case sensitiv ity , white space,

continua tion, and column r ules a pply t o the l ine. After t he sentinel is replaced with

two spaces, initial l ines must hav e a space or zero in column 6 and only white space

a nd n umb ers in columns 1 th rough 5. After t he sentinel is repla ced w ith t wo spa ces,

continuation l ines must hav e a character other than a space or zero in column 6 and

only w hite space in columns 1 t hrough 5. I f t hese criteria a re not met, t he l ine is

t r e a t e d a s a com m e n t a n d i g n or e d .Exa mple: The follow ing forms for specifying conditional compila tion in fi xed source

form are eq uiv alent:

C23456789

!$ 10 IAM = OMP_GET_THREAD_NUM() +

!$ & INDEX

#ifdef _OPENMP

10 IAM = OMP_GET_THREAD_NUM() +

& INDEX

#endif

2.1.3.2 Free Source Form Conditional Compilation Sentinel

The OpenMP Fortran API accepts the following conditional compilation sentinel in

free source form files:

!$

This sentinel can appear in any column as long as i t is preceded only b y white space.

I t m u s t a p pe a r a s a s i n gl e w or d w i t h n o i n t e r v en i n g w h i t e s p a c e. F or t r a n f r e e s ou r ce

form line length, case sensitiv ity , white spa ce, a nd continua tion rules apply to t hel in e . I n i t i a l l i n es m u s t h a v e a s p a ce a f t e r t h e s e n t i n el . C o n t in u e d l in e s m u s t h a v e a n

a mpersand a s the last nonblank char a cter on the l ine, prior to an y comment a ppea ring

on the conditionally compiled l ine. C ontinuation l ines can hav e an ampersand after

t h e s en t i n e l, w i t h op t i on a l w h i t e s p a ce b ef or e a n d a f t e r t h e a m p er s a n d .


396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423424

425

426

427


20/124


Example: The following forms for specifying conditional compilation in free source

form are eq uiv alent:

C23456789!$ IAM = OMP_GET_THREAD_NUM() + &

!$& INDEX

#ifdef _OPENMP

IAM = OMP_GET_THREAD_NUM() + &

INDEX

#endif

2.2 Parallel Region Construct

Th e PARALLEL a n d END PARALLEL directiv es defi ne a parallel region . A pa r a l le l

region is a b lock of code t ha t is to b e executed b y mult iple t hreads in pa ra llel . This is

the fundamental parallel construct in OpenMP that starts parallel execution. These

directiv es ha v e th e following forma t:

!$OMP PARALLEL [clause [[,] clause ]...]

block

!$OMP END PARALLEL

clause can be one of the following:

• PRIVATE(l ist )

• SHARED(l ist )

• DEFAULT(PRIVATE | SHARED | NONE)

• FIRSTPRIVATE(l ist )

• REDUCTION({operator |intrinsic_procedure_name }:l ist )

• COPYIN(l ist )

• IF(scalar_logical_expression )

• NUM_THREADS(scalar_integer_expression )

Th e IF a n d NUM_THREADS clauses are described in this section. The PRIVATE,

SHARED, DEFAULT, FIRSTPRIVATE, REDUCTION, a n d COPYIN cla uses a re describ ed in


2000

428

429

430

431

432

433

434

435

436

437

438

439

440

441

442

443

444

445

446

447

448

449

450

451

452

453

454

455


21/124


Section 2.6.2, page 34. For an example of how to implement coarse-grain parallelism

using th ese directives, see Section A.3, page 64.

W hen a thread encounters a parallel region, i t creates a team of threads, and i tb ecomes the master of the team. The master thread is a memb er of the team. The

numb er of threads in the team is controlled b y env ironment v ariab les, the

NUM_THREADS clau se, an d/or librar y ca lls. F or more informa tion on environment

v aria b les, see C ha pter 4, pa ge 59. For more informa tion on l ibra ry routines, see

C ha pter 3, page 47.

If a para llel region is encount ered w hile dy namic adjustment of the n umb er of

threads is disab led, and the numb er of threads specifi ed for the parallel region

exceeds the numb er th at the ru n-time sy stem can supply, th e b eha v ior of th e progra m

is implementa tion-dependent. An implementat ion ma y, for example, int errupt t he

execution of the program, or i t ma y seria lize th e para llel region.

The numb er of phy sical processors actually hosting the threads at any giv en time isimplementat ion-dependent. Once crea ted, the n umb er of thr eads in the t eam remains

consta nt for t he dura tion of tha t pa ra llel region. I t can b e cha nged either explicitly b y

the user or automatically b y the run- time sy stem from one parallel region to another.

Th e OMP_SET_DYNAMIC l ib r a r y r o u t i n e a n d t h e OMP_DYNAMIC env ironment v a riab le

ca n b e u s ed t o e n a b l e a n d d i sa b l e t h e a u t o m a t i c a d j u st m e n t of t h e n u m b er of t h r e a d s .

For more informa tion on the OMP_SET_DYNAMIC library routine, see S ection 3.1.7,

page 51. For more informa tion on the OMP_DYNAMIC env ironment v aria b le, see

Section 4.3, pa ge 60.

W ithin the dy namic extent of a parallel region, thread numb ers uniq uely identify

each t hread. Thread numb ers a re consecutiv e wh ole numb ers ra nging from zero for

t h e m a s t e r t h r e a d u p t o on e l es s t h a n t h e n u m b er o f t h r e a d s w i t h i n t h e t e a m . Th ev a l u e of t h e t h r e a d n u m b e r i s r e t u r n ed b y a ca l l t o t h e OMP_GET_THREAD_NUM l ib rar y

routine (for more informa tion see S ection 3.1.4, pa ge 49). I f dy na mic threa ds a re

disab led when the parallel region is encountered, and remain disab led until a

sub seq uent, non-nested pa ra llel region is encountered, th en t he t hread numb ers for

t h e t w o r eg i on s a r e con s i st e n t i n t h a t t h e t h r e a d i d en t i fi e d w i t h a g i ve n t h r e a d

numb er in the earlier parallel region will b e identifi ed with the same thread numb er

in the later region.

block denotes a structured b lock of Fortran statements. I t is noncompliant to b ranch

into or out of the b lock . The code conta ined wit hin t he dy na mic extent of the par a llel

region is executed by each thread. The code path can be different for different threads.

Th e END PARALLEL directive denotes th e end of the para llel region. There is a n

implied b arrier at this point. Only the master thread of the team continues execution

past the end of a parallel region.

If a thread in a team executing a parallel region encounters another parallel region, i t

creates a new team, and i t b ecomes the master of that new team. This second parallel

region is ca lled a nested para llel region. B y default , n ested para llel regions a re


456

457

458

459

460

461

462

463

464

465

466

467

468

469

470

471

472

473

474

475

476

477

478

479

480

481

482

483

484

485

486

487

488

489

490

491

492

493

494

495

496


22/124


seria lized; tha t is , they a re executed b y a team composed of one threa d. This default

b ehav ior can b e changed b y using either the OMP_SET_NESTED l ib rary routine or the

OMP_NESTED env ironment v ariab le. For more information on the OMP_SET_NESTED

library routine, see Section 3.1.9, page 52. For more information on the OMP_NESTEDenvironment variable, see Section 4.4, page 61.

I f a n IF clause is present, the enclosed code region is executed in parallel only if the

scalar_logical_expression e va l u a t e s t o .TRUE. . Otherwise, the pa ra llel region is

seria lized. The expression must b e a sca lar Fortra n logical expression. In the a b sence

of a n IF clause, th e region is executed a s i f a n IF(.TRUE.) clause were specified.

Th e NUM_THREADS clause is used to req uest that a specifi c numb er of threads is used

in a parallel region. I t supersedes the numb er of threads indicated b y the

OMP_SET_NUM_THREADS l ib rary routine or the OMP_NUM_THREADS environment

v aria b le for the para llel region i t is a pplied t o. S ub seq uent pa ra llel regions, howev er,

are not af fected unless they hav e their own NUM_THREADS clauses.

scalar_integer_expression must ev aluate to a positiv e scalar integer v alue.

If execution of the program termina tes w hile inside a para llel region, execution of a l l

thr eads t erminat es. All work b efore th e prev ious b a rrier encountered b y th e threa ds

i s g u a r a n t e ed t o b e com p le t ed ; n o n e of t h e w o r k a f t er t h e n e x t b a r r i er t h a t t h e

threads would hav e encountered will hav e b een started. The amount of work done b y

e a ch t h r e a d i n b e t w e en t h e b a r r i er s a n d t h e or d er i n w h i ch t h e t h r e a d s t e r m in a t e a r e

unspecified.

The following restrictions apply to parallel regions:

• Th e PARALLEL/END PARALLEL directiv e pair must appear in the same routine in

the executable section of the code.

• The code enclosed in a PARALLEL/END PARALLEL pair must b e a structured b lock .

It is noncomplia nt to b ra nch into or out of a para llel region.

• O nl y a s in g le IF clause can a ppea r on the directiv e. The IF expression is

ev aluat ed outside th e context of the para llel region. Results a re unspecifi ed i f th e

IF expression conta ins a function reference t ha t ha s side effects.

• O nl y a s in gl e NUM_THREADS clause ca n a ppea r on the directiv e. The NUM_THREADS

expression is ev aluat ed outside t he context of the par a llel region. Results a re

unspecifi ed i f the NUM_THREADS expression contains a function reference th a t ha s

side effects.

• I f t h e d y n a m i c t h r e a d s m e ch a n i s m i s en a b l ed , t h en t h e n u m b er of t h r ea d sreq uested b y the NUM_THREADS cl a u s e i s t h e m a x im u m n u m b er t o u s e in t h e

parallel region.

• Th e or d e r of e va l u a t i on of IF c la u s e s a n d NUM_THREADS clauses is unspecified.


2000

497

498

499

500

501

502

503

504

505

506

507

508

509

510

511

512

513

514

515

516

517

518

519

520

521

522

523

524

525

526

527

528

529

530

531

532

533


23/124


• U n s y n c h r on i z ed u s e of F or t r a n I /O s t a t e m en t s b y m u l t i pl e t h r e a d s o n t h e s a m e

unit ha s unspecifi ed b eha v ior.

2.3 Work-sharing Constructs

A work-sharing construct divides the execution of the enclosed code region among the

memb ers of the tea m tha t encounter i t . A work -sha ring construct must b e enclosed

dy na mica lly with in a para llel region in order for the d irective to execute in para llel .

When a work -sha ring construct is not enclosed dy na mica lly with in a para llel region,

i t i s t r e a t e d a s t h o u gh t h e t h r e a d t h a t e n cou n t e r s i t w e r e a t e a m of s i ze on e . Th e

work -sha ring directiv es do not launch new thr eads, a nd t here is no implied b arr ier on

entry to a work - sharing construct.

The following restrictions apply to the work-sharing directives:

• Wor k -s h a r i n g con s t r u ct s a n d BARRIER directiv es must b e encountered b y a ll

t h r e a d s i n a t e a m o r b y n o n e a t a l l .

• Wor k -s h a r i n g con s t r u ct s a n d BARRIER directives must b e encount ered in t he sa me

o r d e r b y a l l t h r e a d s i n a t e a m .

The follow ing sections d escribe t he w ork-sha ring directives:

• S ection 2.3.1, pa ge 15, describ es the DO a n d END DO d irectives.

• S ection 2.3.2, pa ge 18, describ es the SECTIONS, SECTION, a n d END SECTIONS

directives.

• S ection 2.3.3, page 20, describ es the SINGLE a n d END SINGLE d irectives.

• S ection 2.3.4, pa ge 20, describ es the WORKSHARE a n d END WORKSHARE d irectives.

I f NOWAIT is specifi ed on th e END DO, END SECTIONS, END SINGLE, or

END WORKSHARE directiv e, a n implementa tion ma y omit a ny code to sy nchronize the

t h r e a d s a t t h e e n d of t h e w o r k sh a r i n g c on s t r u ct . I n t h i s ca s e , t h r e a d s t h a t fi n i s h

early ma y proceed st ra ight t o the inst ructions following the w ork -sha ring construct

w i t h o u t w a i t i n g f o r t h e ot h e r m e m b er s o f t h e t e a m t o fi n i s h t h e w or k -s h a r i n g

construct. (S ee S ection A.4, pa ge 64, for a n exa mple with the DO directive.)

2.3.1 DO Directive

Th e DO directiv e specifi es t ha t the i terat ions of t he immediat ely following DO loop

must b e executed in pa ra llel . The loop th a t follows a DO directive cannot be a


534

535

536

537

538

539

540

541

542

543

544

545

546

547

548

549

550

551

552

553

554

555

556

557

558

559

560

561

562

563


24/124


DO WHILE or a DO loop without loop control. The iterations of the DO loop ar e

d i st r i b u t ed a cr os s t h r e a d s t h a t a l r e a d y e x is t .

The format of this directive is as follows:

!$OMP DO [clause [[,] clause ]...]

do_loop

[!$OMP END DO [NOWAIT]]

Th e do_loop m a y b e a do_construct , a n outer _sha r ed_do_constru ct , o r a n

inner_shared_do_construct . A DO construct that contains sev eral DO s t a t e m en t s t h a t

s h a r e t h e s a m e DO termination statement sy ntactically consists of a seq uence of

outer_shared_do_constructs , followed by a single inner_shared_do_construct . I f a n END

DO directive follow s such a DO construct, a DO directive can only be specified for the

fi rst (i .e. , th e outermost) outer _sha r ed_do_constru ct . (See exa mples in Section A.22,

page 81.)


• PRIVATE(l ist )


• LASTPRIVATE(l ist )

• REDUCTION({operator |intrinsic_procedure_name }:l ist )

• SCHEDULE(type [,chunk ])

• ORDERED

Th e SCHEDULE a n d ORDERED clauses are described in this section. The PRIVATE,

FIRSTPRIVATE, LASTPRIVATE, a n d REDUCTION clauses are described in Section

2.6.2, page 34.

If ordered sections are contained in the dy namic extent of the DO directive, the

ORDERED clause must b e present. For more informa tion on ordered sections, see the

ORDERED directive in Section 2.5.6, pa ge 30.

Th e SCHEDULE clause specifi es how iterat ions of t he DO loop are div ided a mong th e

t h r e a d s o f t h e t e a m . chunk must b e a scalar integer expression whose v alue ispositive. The chunk expression is ev alua ted outside t he context of t he DO construct.

Results are unspecifi ed i f the chunk expression contains a function reference tha t ha s

side effects. Within the SCHEDULE(type [,chunk ]) clause sy ntax, type can be one of

the following:


2000

564

565

566

567

568

569

570

571

572

573

574

575

576

577

578

579

580

581

582

583

584

585

586

587

588

589

590

591

592

593

594

595


25/124


Table 1. SCHEDULE C la use Values

type Effect

STATIC When SCHEDULE(STATIC, chunk ) is specifi ed, i tera tions a re div ided

into pieces of a size specified by chunk . The pieces a re sta tically

assigned to threads in the team in a round- rob in fashion in the order

of the thread numb er.

When chunk is not specifi ed, t he i tera tion space is div ided into

contiguous chunk s that are approximately eq ual in size with one

ch u n k a s s ig n ed t o e a c h t h r e a d .

DYNAMIC When SCHEDULE(DYNAMIC,chunk ) is specifi ed, the i terations are

broken into pieces of a size specified by chunk . As e a c h t h r e a d

fi nishes a piece of the i teration space, i t dy namically ob tains the next

set of i terat ions.

When no chunk is specifi ed, i t defaults to 1.

GUIDED When SCHEDULE(GUIDED,chunk ) is specifi ed, the i terat ion spa ce is

divided into pieces such that the size of each successive piece is

exponentially decreasing. chunk specifi es the size of th e sma llest

piece, except possibly the last. The size of the initial piece is

implementa tion-dependent. As each thr ead fi nishes a piece of t he

iteration space, i t dy namically ob tains the next av ailab le piece.

When no chun k is specifi ed, i t defaults to 1.

RUNTIME When SCHEDULE(RUNTIME) is specified, the decision regarding

scheduling is deferred unt il run time. The schedule ty pe a nd chunk

size can b e chosen at run time b y setting the OMP_SCHEDULE

env ironment v ariab le. I f this env ironment v ariab le is not set, the

resulting schedule is implementation-dependent. For more

information on the OMP_SCHEDULE environment variable, see Section

4.1, page 59.

When SCHEDULE(RUNTIME) is specified, it is noncompliant to specify

chunk .

I n t h e a b s en c e of t h e SCHEDULE cla use, the defa ult schedule is

implementation-dependent. An OpenMP-compliant program should not rely on a

part icular schedule for correct execution. Users sh ould not rely on a part icularimplementation of a schedule type for correct execution, because it is possible to have

v aria tions in t he implementa tions of the sa me schedule ty pe a cross dif ferent

compilers.

Threads that complete execution of their assigned loop i terations wait at a b arrier at

t h e END DO directiv e i f the NOWAIT clause is not specified. The functionality of


596

597

598

599

600

601

602

603

604

605

606

607

608

609

610

611

612

613

614

615

616

617

618

619

620

621

622

623

624

625

626

627

628

629

630

631

632

633


26/124


NOWAIT is specified in Section 2.3, page 15. If an END DO directive is not specified, an

END DO directiv e is assumed at the end of the DO loop. If NOWAIT is specifi ed on t he

END DO directive, t he implied FLUSH a t t h e END DO directive is not performed. (See

Section A.4, page 64, for an example of using the NOWAIT cla use. See S ection 2.5.5,page 29, for a description of implied FLUSH.)

P a r a l l e l DO loop control variables are block-level entities within the DO loop. I f t he

loop control v a riab le a lso appea rs in t he LASTPRIVATE l ist of the par a llel DO, i t i s

copied out to a v aria b le of t he sam e na me in t he enclosing PARALLEL region. The

v aria b le in t he enclosing PARALLEL region must b e SHARED i f i t is specifi ed on t he

LASTPRIVATE list of a DO directive.

The following restrictions apply to the DO directives:

• I t i s n on c om p li a n t t o br a n c h ou t of a DO loop associat ed w ith a DO directive.

• The va lues of the loop contr ol pa ra meters of the DO loop a ssociat ed w ith a DOd i r ect i v e m u s t b e t h e s a m e f or a l l t h e t h r e a d s i n t h e t e a m .

• Th e DO loop i tera tion v a riab le must b e of ty pe integer.

• I f u sed , t h e END DO directive must appear immediately a f ter t he end of the loop.

• O n ly a s in gl e SCHEDULE clause can appear on a DO directive.

• O n ly a s in gl e ORDERED clause can appear on a DO directive.

• chunk m ust b e a positiv e sca lar integer expression.

• Th e v a l ue of t h e chunk p a r a m e t er m u s t b e t h e s a m e f or a l l of t h e t h r e a d s i n t h e

t e a m .

2.3.2 SECTIONS Directive

Th e SECTIONS directiv e is a non-iterat iv e work -sha ring construct tha t specifi es tha t

the enclosed sections of code ar e to b e divided a mong threa ds in the t eam. Ea ch

section is executed once b y a threa d in t he tea m.



2000

634

635

636

637

638

639

640

641

642

643

644

645

646

647

648

649

650

651

652

653

654

655

656

657

658

659


27/124


!$OMP SECTIONS [clause [[,] clause ]...]

[!$OMP SECTION]

block

[!$OMP SECTION

block ]

. . .

!$OMP END SECTIONS [NOWAIT]

block denotes a structured b lock of Fortran statements.


• PRIVATE(l ist )


• LASTPRIVATE(l ist )

• REDUCTION({ operator |intrinsic_procedure_name }:l ist )

Th e PRIVATE, FIRSTPRIVATE, LASTPRIVATE, a n d REDUCTION clauses are described

in Section 2.6.2, page 34.

Each section is preceded by a SECTION directiv e, though the SECTION directive isoptiona l for the fi rst section. The SECTION directiv es must appear within the lexical

extent of the SECTIONS/END SECTIONS directiv e pair . The la st section ends a t the

END SECTIONS directive. Threads tha t complete execution of t heir sections wa it a t a

b a r r i e r a t t h e END SECTIONS directiv e i f the NOWAIT clause is not specified. The

functionality of NOWAIT is described in Section 2.3, pa ge 15.

The following restrictions apply to the SECTIONS directive:

• The code enclosed in a SECTIONS/END SECTIONS directive pair must b e a

structured b lock. In a ddition, each constitu ent section m ust a lso be a structured

b lock. I t is noncomplia nt to b ra nch into or out of t he constituent section b locks.

• I t i s n on com p li a n t for a SECTION directiv e to b e outside the lexical extent of theSECTIONS/END SECTIONS directive pair. (See Section A.8, page 67 for an example

tha t uses these directiv es.)


660

661

662

663

664

665

666

667

668

669

670

671

672

673

674

675

676

677

678

679

680

681

682

683

684

685

686

687


28/124


2.3.3 SINGLE Directive

Th e SINGLE directive specifies that the enclosed code is to be executed by only one

t h r e a d i n t h e t e a m . Th r e a d s i n t h e t e a m t h a t a r e n o t e xe cu t i n g t h e SINGLE directivew a i t a t a b a r r i e r a t t h e END SINGLE directiv e i f the NOWAIT clause is not specified.

The fun ctiona lity of NOWAIT is described in Section 2.3, pa ge 15.


!$OMP SINGLE [clause [[,] clause ]...]

block

!$OMP END SINGLE [end_single_modifier ]

w h e r e end_single_modifier is either COPYPRIVATE(l ist )[[,]COPYPRIVATE( l ist )...]or NOWAIT.



• PRIVATE(l ist )

• FIRSTPRIVATE( l i st )

Th e PRIVATE, FIRSTPRIVATE, a n d COPYPRIVATE clauses are described in Section

2.6.2, page 34.

The following restriction applies to the SINGLE directive:

• The code enclosed in a SINGLE/END SINGLE directive pair must b e a structured

b lock. I t is noncompliant to b ra nch into or out of the b lock.

See Section A.9, page 67, for an example of the SINGLE directive.

The following restriction applies to the END SINGLE directive:

• S p eci fi c a t i on of b ot h a COPYPRIVATE cl a u s e a n d a NOWAIT cl a u s e on t h e s a m e

END SINGLE directive is noncompliant.

2.3.4 WORKSHARE Directive

Th e WORKSHARE directive divides th e w ork of executin g t he enclosed code int o sepa ra te

u n i t s o f w or k , a n d ca u s e s t h e t h r e a d s o f t h e t e a m t o s h a r e t h e w o r k o f ex ecu t i n g t h e

enclosed code such that each unit is executed only once. The units of work may be

assigned to threads in any manner as long as each unit is executed exactly once.


2000

688

689

690

691

692

693

694

695

696

697

698

699

700

701

702

703

704

705

706

707

708

709

710

711

712

713

714

715

716


29/124


!$OMP WORKSHARE

block

!$OMP END WORKSHARE [NOWAIT]

A BARRIER is implied follow ing th e enclosed code if th e NOWAIT clause is not specified

on t h e END WORKSHARE directive. The functionality of NOWAIT is described in Section

2.3, page 15. An implementation of the WORKSHARE directiv e must insert any

s y n ch r on i z a t i on t h a t i s r eq u i r ed t o m a i n t a i n s t a n d a r d F or t r a n s e m a n t i cs . F or

exa mple, th e effects of one sta tement with in block must a ppea r to occur b efore th e

execution of succeeding statements, and the ev aluation of the r ight hand side of an

assignment must appear to hav e b een completed prior to the effects of assigning to

the left hand side.

The sta tements in block a re div ided into units of w ork as follows:

• F o r a r r a y ex pr e ss i on s w i t h i n ea c h st a t e m en t , in c lu d i n g t r a n s f or m a t i on a l a r r a y

intrinsic functions that compute scalar v alues from array s:

– E v a l u a t i on o f ea c h e le m en t of t h e a r r a y e xp r es s ion i s a u n i t of w o r k .

– E v a l u a t i on o f t r a n s f or m a t i on a l a r r a y i n t r i n si c f u n ct i on s m a y b e fr e el y

sub divided int o an y numb er of units of work .

• I f a WORKSHARE d i r ect i v e is a p pl ie d t o a n a r r a y a s s i g n m en t s t a t e m en t , t h e

assignment of each element is a unit of work .

• I f a WORKSHARE directiv e is applied to a scalar assignment statement, the

assignment operation is a single unit of work .

• I f a WORKSHARE directiv e is a pplied to a reference to a n elementa l function,

application of the function to the corresponding elements of any array argument is

t r e a t e d a s a u n i t of w o r k . H e n ce , if a n y a c t u a l a r g u m en t i n a r e fe r en ce t o a n

e le m en t a l f u n ct i on i s a n a r r a y , t h e r e fe r en ce i s t r e a t e d i n t h e s a m e w a y a s i f t h e

function ha d b een a pplied separa tely to corresponding elements of ea ch a rra y

a c t u a l a r g u m en t .

• I f a WORKSHARE directive is applied to a WHERE s tatement or construct, the

e va l u a t i on of t h e m a s k e xp r es s ion a n d t h e m a s k ed a s s i g n m en t s a r e w o r ks h a r e d .

• I f a WORKSHARE directive is applied to a FORALL s tatement or construct, the

evaluation of the mask expression, expressions occurring in the specification of the

i t e ra t i on s p a c e, a n d t h e m a s k ed a s s ig n m en t s a r e w or k s h a r e d .

• F or ATOMIC directives a nd t heir corresponding assignments, the updat e of each

scalar v ariab le is a single unit of work .

• F or CRITICAL constructs, each constru ct is a single unit of w ork.


717

718

719

720

721

722

723

724

725

726

727

728

729

730

731

732

733

734

735

736

737

738

739

740

741

742

743

744

745

746

747

748

749

750

751


30/124


• F or PARALLEL constructs, each constru ct is a single unit of w ork with respect to

t h e WORKSHARE construct. The sta tements conta ined in PARALLEL constructs a re

executed b y new t eams of threads formed for each PARALLEL directive.

• I f n on e of t h e r u le s a b o ve a p p ly t o a p or t i on of a s t a t e m en t i n block , t h en t h a t

portion is a single unit of work .

The transformational array intrinsic functions are MATMUL, DOT_PRODUCT, SUM,

PRODUCT, MAXVAL, MINVAL, COUNT, ANY, ALL, SPREAD, PACK, UNPACK, RESHAPE,

TRANSPOSE, EOSHIFT, CSHIFT, MINLOC, a n d MAXLOC.

I f a n a rra y expression in the b lock references t he v a lue, a ssocia tion sta tus, or

allocation status of PRIVATE v aria b les, th e v a lue of t he expression is undefi ned,

unless the same v alue would b e computed b y ev ery thread.

I f a n a r r a y a s s ig n m e n t , a s ca l a r a s s i g n m en t , a m a s k e d a r r a y a s s i g n m en t , or a FORALL

assignment assigns to a priv ate v ariab le in the b lock , the result is unspecifi ed.

Th e WORKSHARE directiv e causes t he sha ring of w ork to occur only in t he lexically

enclosed block.

The following restrictions apply to the WORKSHARE directive:

• block ma y conta in st a tements wh ich b ind to lexica lly enclosed PARALLEL

con s t r u ct s . S t a t e m en t s i n t h e s e PARALLEL constructs a re not restricted.

• block m a y c on t a i n ATOMIC directiv es a nd CRITICAL constructs.

• block m u s t on l y con t a i n a r r a y a s s ig n m en t s t a t e m en t s , s ca l a r a s s ig n m e n t

s t a t e m e n t s , FORALL s t a t e m en t s , FORALL constructs, WHERE s t a t e m en t s , o r WHERE

constructs.

• block must not contain any user defi ned function calls unless the function is

ELEMENTAL.

• The code enclosed in a WORKSHARE/END WORKSHARE directive pair must b e a

structured b lock. I t is noncomplia nt to b ra nch into or out of t he b lock.

2.4 Combined Parallel Work-sharing Constructs

The comb ined pa ra llel w ork-sha ring constructs a re shortcuts for specify ing a para llel

region t ha t conta ins only one work -sha ring construct. The sema ntics of these

directives are identical to that of explicitly specifying a PARALLEL directive followed

b y a single work -sha ring construct.

The following sections describe the combined parallel work-sharing directives:


2000

752

753

754

755

756

757

758

759

760

761

762

763

764

765

766

767

768

769

770

771

772

773

774

775

776

777

778

779

780

781

782

783


31/124


• S ection 2.4.1, pa ge 23, describ es the PARALLEL DO a n d END PARALLEL DO

directives.

• S ection 2.4.2, pa ge 24, describ es the PARALLEL SECTIONS a n dEND PARALLEL SECTIONS d irectives.

• S ection 2.4.3, page 24, describ es the PARALLEL WORKSHARE a n d

END PARALLEL WORKSHARE d irectives.

2.4.1 PARALLEL DO Directive

Th e PARALLEL DO directive provides a shortcut form for specifying a parallel region

t h a t c o n t a i n s a s i n g l e DO directive. (See Section A.1, page 63, for an example.)


!$OMP PARALLEL DO [clause [[,] clause ]...]

do_loop

[!$OMP END PARALLEL DO]

Th e do_loop m a y b e a do_construct , a n outer _sha r ed_do_constru ct , or a n

inner_shared_do_construct . A DO construct that contains sev eral DO s t a t e m en t s t h a t

s h a r e t h e s a m e DO terminat ion st a tement sy n ta ctica lly consists of a seq uence of

outer_shared_do_constructs , followed by a single inner_shared_do_construct . I f a n ENDPARALLEL DO directive follow s such a DO construct, a PARALLEL DO directive can

only be specified for the first (i.e. , the outermost) outer _sha r ed_do_constr uct . (See

Section A.22, page 81, for examples.)

clause ca n b e one of t he cla uses a ccepted b y either the PARALLEL or t h e DO directive.

For more information ab out the PARALLEL directiv e and the IF a n d NUM_THREADS

clauses, see Section 2.2, page 12. For more information about the DO directiv e a nd t he

SCHEDULE a n d ORDERED clauses, see Section 2.3.1, page 15. For more information on

the remaining clauses, see Section 2.6.2, page 34.

I f t h e END PARALLEL DO directive is not specified, the PARALLEL DO en d s w i t h t h e

DO loop tha t immediately follows the PARALLEL DO directive. I f used, th e

END PARALLEL DO directiv e must appear immediately af ter the end of the DO loop.

The semantics are identical to explicitly specifying a PARALLEL directive immediately

followed by a DO directive.


784

785

786

787

788

789

790

791

792

793

794

795

796

797

798

799

800

801

802

803

804

805

806

807

808

809

810

811

812

813


32/124


2.4.2 PARALLEL SECTIONS Directive

Th e PARALLEL SECTIONS directive provides a shortcut form for specifying a parallel

region that contains a single SECTIONS directive. The sema ntics a re identica l t oexplicitly specifying a PARALLEL directive immediately followed by a SECTIONS

directive.


!$OMP PARALLEL SECTIONS [clause [[,] clause ]...]

[!$OMP SECTION ]

block

[!$OMP SECTION

block ]

. . .

!$OMP END PARALLEL SECTIONS


clause ca n b e one of the cla uses a ccepted b y either the PARALLEL or t he SECTIONS

directiv e. For more information ab out the PARALLEL directiv e and the IF a n d

NUM_THREADS clauses, see S ection 2.2, pa ge 12. For more informa tion a b out theSECTIONS directive, see Section 2.3.2, page 18. For more information on the

remaining clauses, see Section 2.6.2, page 34.

The last section ends at the END PARALLEL SECTIONS directive.

2.4.3 PARALLEL WORKSHARE Directive

Th e PARALLEL WORKSHARE directive provides a shortcut form for specifying a

parallel region that contains a single WORKSHARE directiv e. The sema ntics a re

identical to explicitly specifying a PARALLEL directive immediately followed by a

WORKSHARE directive.



2000

814

815

816

817

818

819

820

821

822

823

824

825

826

827

828

829

830

831

832

833

834

835

836

837

838

839


33/124


!$OMP PARALLEL WORKSHARE [clause [[,] clause ]...]

block

!$OMP END PARALLEL WORKSHARE


clause ca n b e one of t he cla uses a ccepted b y either the PARALLEL or t h e WORKSHARE

directiv e. For more information ab out the PARALLEL d i r ect i v e a n d t h e IF a n d

NUM_THREADS cla uses, see S ection 2.2, pa ge 12. For more informa tion a b out the

remaining clauses, see Section 2.3.4, page 20.

2.5 Synchronization Constructs and the MASTER Directive

The following sections describe the synchronization constructs and the MASTER

directive:

• S ection 2.5.1, pa ge 25, describ es the MASTER a n d END MASTER d irectives.

• S ection 2.5.2, pa ge 26, describ es the CRITICAL a n d END CRITICAL d irectives.

• S ection 2.5.3, pa ge 26, describ es the BARRIER directive.

• S ection 2.5.4, pa ge 27, describ es the ATOMIC directive.

• S ection 2.5.5, pa ge 29, describ es the FLUSH directive.

• S ection 2.5.6, page 30, describ es the ORDERED a n d END ORDERED d irectives.

2.5.1 MASTER Directive

The code enclosed within MASTER a n d END MASTER directives is executed by the

m a s t e r t h r e a d o f t h e t e a m .


!$OMP MASTER

block

!$OMP END MASTER


840

841

842

843

844

845

846

847

848

849

850

851

852

853

854

855

856

857

858

859

860

861

862

863


34/124


The other t hreads in t he t eam sk ip the enclosed section of code a nd continue

execution. There is no implied b ar rier either on entry to or exit from the ma ster

section.

The following restriction applies to the MASTER directive:

• The code enclosed in a MASTER/ END MASTER directiv e pair must b e a structured

b lock. I t is noncomplia nt to b ra nch into or out of the b lock.

2.5.2 CRITICAL Directive

Th e CRITICAL a n d END CRITICAL directives restrict access to the enclosed code to

on l y on e t h r e a d a t a t i m e .


!$OMP CRITICAL [(n a m e )]

block

!$OMP END CRITICAL [(n a m e )]

The optional n a m e argument identifi es the critical section.

A thread waits at the b eginning of a critical section until no other thread is executing

a cr i t i ca l s e ct i on w i t h t h e s a m e n a m e . Al l u n n a m e d CRITICAL directiv es ma p t o the

s a m e n a m e . C r i t i ca l s ect i on n a m e s a r e g l ob a l e n t i t i es o f t h e p r og r a m . I f a n a m econflicts with any other entity , the b ehav ior of the program is unspecifi ed.

The following restrictions apply to the CRITICAL directive:

• The code enclosed in a CRITICAL/END CRITICAL directiv e pair must b e a

structured b lock. I t is noncompliant to b ra nch int o or out of the b lock.

• I f a n a m e is specified on a CRITICAL directive, the sa me n a m e m u s t a l s o b e

specifi ed on t he END CRITICAL directiv e. I f n o n a m e a p pe a r s o n t h e CRITICAL

directive, no n a m e ca n a p p ea r on t h e END CRITICAL directive.

S ee S ection A.5, page 64, for a n exam ple tha t uses na med CRITICAL sections.

2.5.3 BARRIER Directive

Th e BARRIER directive sy nchronizes a ll the t hreads in a tea m. When encountered,

e a ch t h r e a d w a i t s u n t i l a l l of t h e o t h er t h r e a d s i n t h a t t e a m h a v e r ea c h ed t h i s p oi n t .


2000

864

865

866

867

868

869

870

871

872

873

874

875

876

877

878

879

880

881

882

883

884

885

886

887

888

889

890

891


35/124



!$OMP BARRIER

The following restrictions apply to the BARRIER directive:

• Wor k -s h a r i n g con s t r u ct s a n d BARRIER directiv es must b e encountered b y a ll

t h r e a d s i n a t e a m o r b y n o n e a t a l l .

• Wor k -s h a r i n g con s t r u ct s a n d BARRIER directives must b e encount ered in t he sa me

o r d e r b y a l l t h r e a d s i n a t e a m .

2.5.4 ATOMIC Directive

Th e ATOMIC directive ensures tha t a specifi c memory location is upda ted a tomica lly,

rather than exposing i t to the possib il i ty of multiple, simultaneous writing threads.


!$OMP ATOMIC

This directiv e a pplies only to th e immediately following sta tement, w hich m ust ha v e

one of the following forms:

x = x operat or expr

x = expr operat or x

x = intrinsic_procedure_name (x , expr_list )

x = intrinsic_procedure_name (expr_list , x )

In the preceding statements:

• x is a scalar v ariab le of intrinsic ty pe.

• expr is a sca lar expression t ha t does not reference x .• expr_list is a comma -sepa ra ted, non-empty l ist of scala r expressions tha t do not

reference x . When intrinsic_procedure_name refers t o IAND, IOR, or IEOR, exactly

one expression must a ppea r in expr_list .

• intrinsic_procedure_name is one of MAX, MIN, IAND, IOR, or IEOR.


892

893

894

895

896

897

898

899

900

901

902

903

904

905

906

907

908

909

910

911

912

913

914

915

916


36/124


• operator is one of +, *, -, /, .AND., .OR., .EQV., or .NEQV. .

• Th e op er a t or s i n expr m u s t h a v e p r ece d en ce e q u a l t o o r g r e a t e r t h a n t h e

precedence of operator , x operator expr m u s t b e m a t h e m a t i ca l l y e q u iv a l en t t o x opera tor (expr) , a n d expr operator x m u s t b e m a t h e m a t i ca l l y

fspec20 bars

Documents