Upload File

hc td51021 loop transformations motivation loop level transformations catalogus –loop merging...

  • Home
  • Documents
  • HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling
69
HC TD5102 1 Loop Transformations • Motivation • Loop level transformations catalogus – Loop merging – Loop interchange – Loop unrolling – Unroll-and-Jam – Loop tiling • Loop Transformation Theory and Dependence Analysis Thanks for many slides go to the DTSE people from IMEC an Dr. Peter Knijnenburg from Leiden University

Upload: hester-hall

Post on 18-Dec-2015

261 views

Category:

Documents


0 download

Report

Tags:

TRANSCRIPT

Page 1: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 1

Loop Transformations• Motivation• Loop level transformations catalogus

– Loop merging– Loop interchange– Loop unrolling– Unroll-and-Jam – Loop tiling

• Loop Transformation Theory and Dependence Analysis

Thanks for many slides go to the DTSE people from IMEC and

Dr. Peter Knijnenburg from Leiden University

Page 2: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 2

Loop Transformations

• Change the order in which the iteration space is traversed.

• Can expose parallelism, increase available ILP, or improve memory behavior.

• Dependence testing is required to check validity of transformation.

Page 3: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 3

Why loop trafos: examples

Example 1: in-place mapping

for (j=0; j<M; j++) for (i=0; i<N; i++) A[i][j] = f(A[i][j-1]);

for (i=0; i<N; i++) OUT[i] = A[i][M-1];

for (i=0; i<N; i++){ for (j=0; j<M; j++) A[i][j] = f(A[i][j-1]); OUT[i] = A[i][M-1];}

OUT

j

i

A

Page 4: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 4

Why loop trafos: examples

for (i=0; i<N; i++) B[i] = f(A[i]);for (i=0; i<N; i++) C[i] = g(B[i]);

for (i=0; i<N; i++){ B[i] = f(A[i]); C[i] = g(B[i]);}

N cyc.

N cyc.2N cyc.

Example 2: memory allocation

2 background ports 1 backgr. + 1 foregr. ports

CPU

Background memory

Foreground

memory

Externalmemoryinterface

Page 5: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 5

Loop transformation catalogue• merge

– improve locality

• bump• extend/reduce• body split• reverse

– improve regularity

• interchange– improve

locality/regularity

• skew• tiling• index split• unroll• unroll-and-jam

Page 6: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 6

Loop Merge

• Improve locality• Reduce loop overhead

for Ia = exp1 to exp2 A(Ia)for Ib = exp1 to exp2

B(Ib)

for I = exp1 to exp2

A(I) B(I)

Page 7: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 7

Loop Merge:Example of locality improvement

Consumptions of second loop closer to productions and consumptions of first loop

• Is this always the case?

for (i=0; i<N; i++) B[i] = f(A[i]); C[i] = f(B[i],A[i]);

for (i=0; i<N; i++) B[i] = f(A[i]);for (j=0; j<N; j++) C[j] = f(B[j],A[j]);

Location

Time

Production/Consumption

Consumption(s)

Location

Time

Production/Consumption

Consumption(s)

Page 8: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 8

Loop Merge:Satisfy dependencies• Data dependencies from first to second loop

can block Loop Merge• Dependency is allowed if

I: cons(I) prod(I) in loop 2• Enablers: Bump, Reverse, Skew

for (i=0; i<N; i++) B[i] = f(A[i]);

for (i=0; i<N; i++) C[i] = g(B[N-1]);

for (i=0; i<N; i++) B[i] = f(A[i]);

for (i=2; i<N; i++) C[i] = g(B[i-2]);

N-1 >= i i-2 < i

Page 9: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 9

for I = exp1 to exp2 A(I)

for I = exp1+N to exp2+N A(I-N)

Loop Bump

Page 10: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 10

Loop Bump:Example as enablerfor (i=2; i<N; i++) B[i] = f(A[i]);for (i=0; i<N-2; i++) C[i] = g(B[i+2]);

i+2 > i merging not possible

Loop Bump for (i=2; i<N; i++)

B[i] = f(A[i]);for (i=2; i<N; i++) C[i-2] = g(B[i+2-2]);

i+2–2 = i merging possible

Loop Merge

for (i=2; i<N; i++) B[i] = f(A[i]); C[i-2] = g(B[i]);

Page 11: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 11

for I = exp1 to exp2 A(I)

for I = exp3 to exp4 if Iexp1 and Iexp2

A(I)

Loop Extend

exp3 exp1 exp4 exp2

Page 12: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 12

Loop Extend:Example as enablerfor (i=0; i<N; i++) B[i] = f(A[i]);for (i=2; i<N+2; i++) C[i-2] = g(B[i]);

Loop Extend

for (i=0; i<N+2; i++) if(i<N) B[i] = f(A[i]);for (i=0; i<N+2; i++) if(i>=2) C[i-2] = g(B[i);

Loop Merge

for (i=0; i<N+2; i++) if(i<N) B[i] = f(A[i]); if(i>=2) C[i-2] = g(B[i);

Page 13: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 13

for I = max(exp1,exp3) to min(exp2,exp4) A(I)

Loop Reduce

for I = exp1 to exp2 if Iexp3 and Iexp4

A(I)

Page 14: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 14

for I = exp1 to exp2

A(I)B(I)

for Ia = exp1 to exp2 A(Ia)

for Ib = exp1 to exp2 B(Ib)

Loop Body Split

A(I) must besingle-assignment

Page 15: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 15

Loop Body Split:Example as enabler

for (i=0; i<N; i++) A[i] = f(A[i-1]); B[i] = g(in[i]);for (j=0; j<N; j++) C[i] = h(B[i],A[N]);

for (i=0; i<N; i++) A[i] = f(A[i-1]);for (k=0; k<N; k++) B[k] = g(in[k]);for (j=0; j<N; j++) C[j] = h(B[j],A[N]);

Loop Body Split

for (i=0; i<N; i++) A[i] = f(A[i-1]);for (j=0; j<N; j++) B[j] = g(in[j]); C[j] = h(B[j],A[N]);

Loop Merge

Page 16: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 16

for I = exp1 to exp2 A(I)

for I = exp2 downto exp1

A(I)

for I = exp1 to exp2

A(exp2-(I-exp1))

Loop Reverse

OR

Location

Time

Location

Time

ProductionConsumption

Page 17: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 17

Loop Reverse:Satisfy dependencies

No loop-carried dependencies allowed !

for (i=0; i<N; i++) A[i] = f(A[i-1]);

Enabler: data-flow transformations (associative)

A[0] = …for (i=1; i<=N; i++) A[i]= A[i-1]+ f(…);

… A[N] … A[N] = …for (i=N-1; i>=0; i--) A[i]= A[i+1]+ f(…);

… A[0] …

Page 18: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 18

Loop Reverse:Example as enablerfor (i=0; i<=N; i++) B[i] = f(A[i]);for (i=0; i<=N; i++) C[i] = g(B[N-i]);

N–i > i merging not possible

Loop Reverse for (i=0; i<=N; i++)

B[i] = f(A[i]);for (i=0; i<=N; i++) C[N-i] = g(B[N-(N-i)]);

N-(N-i) = i merging possible

Loop Merge for (i=0; i<=N; i++)

B[i] = f(A[i]); C[N-i] = g(B[i]);

Page 19: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 19

Loop Interchange

for I1 = exp1 to exp2 for I2 = exp3 to exp4

A(I1, I2)

for I2 = exp3 to exp4 for I1 = exp1 to exp2

A(I1, I2)

Page 20: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 20

Loop Interchange

LoopInterchange

for(i=0; i<W; i++) for(j=0; j<H; j++) A[i][j] = …;

for(j=0; j<H; j++) for(i=0; i<W; i++) A[i][j] = …;

i

j

i

j

Page 21: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 21

Loop Interchange

• Validity: dependence direction vectors.

• Mostly used to improve cache behavior.

• The innermost loop (loop index changes fastest) should (only) index the right-most array index expression in case of row-major storage like in C.

• Can improve execution time by 1 or 2 orders of magnitude.

Page 22: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 22

Loop Interchange (cont’d)

• Loop interchange can also expose parallelism.

• If an inner-loop does not carry a dependence (entry in direction vector equals ‘=‘), this loop can be executed in parallel.

• Moving this loop outwards increases the granularity of the parallel loop iterations.

Page 23: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 23

Loop Interchange:Example of locality improvement

• Productions and consumptionsas local and regular as possible

• Exploration required

for (j=0; j<M; j++) for (i=0; i<N; i++) B[i][j] = f(A[j],B[i][j-1]);

for (i=0; i<N; i++) for (j=0; j<M; j++) B[i][j] = f(A[j],B[i][j-1]);

Page 24: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 24

Loop Interchange:Satisfy dependencies• No loop-carried dependencies allowed

unless I2: prod(I2) cons(I2)

for (i=0; i<N; i++) for (j=0; j<M; j++) A[i][j] = f(A[i-1][j+1]);

• Enablers: – data-flow transformations– Loop Bump

for (i=0; i<N; i++) for (j=0; j<M; j++) A[i][j] = f(A[i-1][j]);

Page 25: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 25

for I1 = exp1 to exp2

for I2 = exp3+.I1+ to exp4+.I1+ A(I1, I2-.I1- )

for I1 = exp1 to exp2

for I2 = exp3+.I1+ to exp4+.I1+ A(I1, I2-.I1- )

for I1 = exp1 to exp2

for I2 = exp3 to exp4

A(I1, I2)

Loop Skew:Basic transformation

Page 26: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 26

Loop Skew

LoopSkewing

for(j=0;j<H;j++) for(i=0;i<W;i++) A[i][j] = ...;

for(j=0; j<H; j++) for(i=0+j; i<W+j;i++) A[I-j][j] = ...;

i

j

i

j

Page 27: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 27

for (i=0; i<N; i++) for (j=0; j<M; j++) … = f(A[i+j]);

Loop Skew: Example as enabler of regularity improvement

Loop Skew for (i=0; i<N; i++)

for (j=i; j<i+M; j++) … = f(A[j]);

for (j=0; j<N+M; j++) for (i=0; i<N; i++) if (j>=i && j<i+M) … = f(A[j]);

Loop Extend &

Interchange

Page 28: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 28

for I = 0 to exp1 . exp2

A(I)

for I1 = 0 to exp2

for I2 = exp1.I1 to exp1.(I1 +1)

A(I2)

Loop Tiling

Tile size exp1

Tile factor exp2

Page 29: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 29

Loop Tiling

LoopTiling

j

i

i

for(i=0;i<9;i++) A[i] = ...;

for(j=0; j<3; j++) for(i=4*j; i<4*j+4; i++) if (i<9) A[i] = ...;

Page 30: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 30

Loop Tiling

• Improve cache reuse by dividing the iteration space into tiles and iterating over these tiles.

• Only useful when working set does not fit into cache or when there exists much interference.

• Two adjacent loops can legally be tiled if they can legally be interchanged.

Page 31: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 31

2-D Tiling Example

for(i=0; i<N; i++)

for(j=0; j<N; j++)

A[i][j] = B[j][i];

for(TI=0; TI<N; TI+=16)

for(TJ=0; TJ<N; TJ+=16)

for(i=TI; i<min(TI+16,N);i++)

for(j=TJ; j<min(TJ+16,N); j++)

A[i][j] = B[j][i];

Page 32: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 32

Selecting a Tile Size

• Current tile size selection algorithms use a cache model:– Generate collection of tile sizes;– Estimate resulting cache miss rate;– Select best one.

• Only take into account L1 cache.

• Mostly do not take into account n-way associativity.

Page 33: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 33

for Ia = exp1 to exp2

A(Ia)

for Ia = exp1 to pA(Ia)

for Ib = p+1 to exp2

A(Ib)

Loop Index Split

Page 34: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 34

Loop Unrolling

• Duplicate loop body and adjust loop header.

• Increases available ILP, reduces loop overhead, and increases possibilities for common subexpression elimination.

• Always valid.

Page 35: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 35

Loop Unrolling: Downside

• If unroll factor is not divisor of trip count, then need to add remainder loop.

• If trip count not known at compile time, need to check at runtime.

• Code size increases which may result in higher I-cache miss rate.

• Global determination of optimal unroll factors is difficult.

Page 36: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 36

for I = exp1 to exp2

A(I)

A(exp1)A(exp1+1)A(exp1+2)…A(exp2)

(Partial) Loop Unrolling

for I = exp1/2 to exp2 /2 A(2l)A(2l+1)

Page 37: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 37

for (L=1; L < 4; L++) for (i=0; i < (1<<L); i++) A(L,i);

for (i=0; i < 2; i++) A(1,i);for (i=0; i < 4; i++) A(2,i);for (i=0; i < 8; i++) A(3,i);

Loop Unroll: Example of removal of non-affine iterator

Page 38: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 38

Unroll-and-Jam

• Unroll outerloop and fuse new copies of the innerloop.

• Increases size of loop body and hence available ILP.

• Can also improve locality.

Page 39: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 39

Unroll-and-Jam Example

• More ILP exposed• Spatial locality of B enhanced

for (i=0;i<N;i++)

for (j=0;j<N;j++)

A[i][j] = B[j][i];

for (i=0; i<N; i+=2)

for (j=0; j<N; j++) {

A[i][j] = B[j][i];

A[i+1][j] =B[j][i+1];

}

Page 40: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 40

Simplified loop transformation script

• Give all loops same nesting depth– Use one-iteration dummy loops if necessary

• Improve regularity – Usually applying loop interchange or reverse

• Improve locality– Usually applying loop merge– Break data dependencies with loop bump/skew– Sometimes loop index split or unrolling is easier

Page 41: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 41

Loop transformation theory

• Iteration spaces

• Polyhedral model

• Dependency analysis

Page 42: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 42

Technical Preliminaries (1)do i = 2, Ndo j = i, N x[i,j]=0.5*(x[x[i-1,ji-1,j]]+ x[i,j-1])

enddo

i

j

2 3 4

2

3

4

0

1

10

01

j

i

(1)

• Address expr (1)

• perfect loop nest• iteration space • dependence vector

Page 43: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 43

j

i

n

m

01

10

Technical Preliminaries (2)Switch loop indexes:do m = 2, Ndo n = 2, m x[n,m]=0.5*(x[n-1,m]x[n-1,m]+ x[n,m-1])

enddo

affine transformation

0

1

01

10

0

1

01

10

10

01

0

1

10

01

n

m

n

m

j

i

(2)

(2)

Page 44: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 44

Polyhedral Model

• Polyhedron is set x : Ax c for some matrix A and bounds vector c

• Polyhedra (or Polytopes) are objects in a many-dimensional space without holes

• Iteration spaces of loops (with unit stride) can be represented as polyhedra

• Array accesses and loop transformations can be represented as matrices

Page 45: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 45

Iteration Space• A loop nest is represented as BI b for

iteration vector I• Example: for(i=0; i<10;i++) -1 0 0 for(j=i; j<10;j++) 1 0 i 9 1 -1 j 0 0 1 9

Page 46: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 46

Array Accesses

• Any array access A[e1][e2] for linear index expressions e1 and e2 can be represented as an access matrix and offset vector.

A + a

• This can be considered as a mapping from the iteration space into the storage space of the array (which is a trivial polyhedron)

Page 47: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 47

Unimodular Matrices

• A unimodular matrix T is a matrix with integer entries and determinant 1.

• This means that such a matrix maps an object onto another object with exactly the same number of integer points in it.

• Its inverse T¹ always exist and is unimodular as well.

Page 48: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 48

Types of Unimodular Transformations

• Loop interchange

• Loop reversal

• Loop skewing for arbitrary skew factor

• Since unimodular transformations are closed under multiplication, any combination is a unimodular transformation again.

Page 49: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 49

Application

• Transformed loop nest is given by AT¹ I’ a• Any array access matrix is transformed into

AT¹.• Transformed loop nest needs to be

normalized by means of Fourier-Motzkin elimination to ensure that loop bounds are affine expressions in more outer loop indices.

Page 50: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 50

Dependence Analysis

Consider following statements:

S1: a = b + c;

S2: d = a + f;

S3: a = g + h;

• S1 S2: true or flow dependence = RaW

• S2 S3: anti-dependence = WaR

• S1 S3: output dependence = WaW

Page 51: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 51

Dependences in Loops

• Consider the following loop

for(i=0; i<N; i++){

S1: a[i] = …;

S2: b[i] = a[i-1];}

• Loop carried dependence S1 S2.

• Need to detect if there exists i and i’ such that i = i’-1 in loop space.

Page 52: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 52

Definition of Dependence

• There exists a dependence if there two statement instances that refer to the same memory location and (at least) one of them is a write.

• There should not be a write between these two statement instances.

• In general, it is undecidable whether there exist a dependence.

Page 53: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 53

Direction of Dependence

• If there is a flow dependence between two statements S1 and S2 in a loop, then S1 writes to a variable in an earlier iteration than S2 reads that variable.

• The iteration vector of the write is lexicographically less than the iteration vector of the read.

• I I’ iff i1 = i’1 i(k-1) = i’(k-1) ik < i’k for some k.

Page 54: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 54

Direction Vectors

• A direction vector is a vector

(=,=,,=,<,*,*,,*)– where * can denote = or < or >.

• Such a vector encodes a (collection of) dependence.

• A loop transformation should result in a new direction vector for the dependence that is also lexicographically positive.

Page 55: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 55

Loop Interchange

• Interchanging two loops also interchanges the corresponding entries in a direction vector.

• Example: if direction vector of a dependence is (<,>) then we may not interchange the loops because the resulting direction would be (>,<) which is lexicographically negative.

Page 56: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 56

Affine Bounds and Indices

• We assume loop bounds and array index expressions are affine expressions:

a0 + a1 * i1 + + ak * ik

• Each loop bound for loop index ik is an affine expressions over the previous loop indices i1 to i(k-1).

• Each loop index expression is a affine expression over all loop indices.

Page 57: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 57

Non-Affine Expressions

• Index expressions like i*j cannot be handled by dependence tests. We must assume that there exists a dependence.

• An important class of index expressions are indirections A[B[i]]. These occur frequently in scientific applications (sparse matrix computations).

• In embedded applications???

Page 58: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 58

Linear Diophantine Equations

• A linear diophantine equations is of the form

aj * xj = c

• Equation has a solution iff gcd(a1,,an) is divisor of c

Page 59: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 59

GCD Test for Dependence

• Assume single loop and two references A[a+bi] and A[c+di].

• If there exist a dependence, then gcd(b,d) divides (c-a).

• Note the direction of the implication!

• If gcd(b,d) does not divide (c-a) then there exists no dependence.

Page 60: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 60

GCD Test (cont’d)

• However, if gcd(b,d) does divide (c-a) then there might exist a dependence.

• Test is not exact since it does not take into account loop bounds.

• For example:

• for(i=0; i<10; i++)

• A[i] = A[i+10] + 1;

Page 61: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 61

GCD Test (cont’d)

• Using the Theorem on linear diophantine equations, we can test in arbitrary loop nests.

• We need one test for each direction vector.

• Vector (=,=,,=,<,) implies that first k indices are the same.

• See book by Zima for details.

Page 62: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 62

Other Dependence Tests

• There exist many dependence test– Separability test– GCD test– Banerjee test– Range test– Fourier-Motzkin test– Omega test

• Exactness increases, but so does the cost.

Page 63: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 63

Fourier-Motzkin Elimination

• Consider a collection of linear inequalities over the variables i1,,in

• e1(i1,,in) e1’(i1,,in)• • em(i1,,in) em’(i1,,in)

• Is this system consistent, or does there exist a solution?

• FM-elimination can determine this.

Page 64: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 64

FM-Elimination (cont’d)

• First, create all pairs L(i1,,i(n-1)) in and in U(i1,,i(n-1)). This is solution for in.

• Then create new system

• L(i1,,i(n-1)) U(i1,,i(n-1))

• together with all original inequalities not involving in.

• This new system has one variable less and we continue this way.

Page 65: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 65

FM-Elimination (cont’d)

• After eliminating i1, we end up with collection of inequalities between constants c1 c1’.

• The original system is consistent iff every such inequality can be satisfied.

• There does not exist an inequality like • 10 3.• There may be exponentially many new

inequalities generated!

Page 66: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 66

Fourier-Motzkin Test

• Loop bounds plus array index expressions generate sets of inequalities, using new loop indices i’ for the sink of dependence.

• Each direction vector generates inequalities

• i1 = i1’ i(k-1) = i(k-1)’ ik < ik’

• If all these systems are inconsistent, then there exists no dependence.

• This test is not exact (real solutions but no integer ones) but almost.

Page 67: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 67

N-Dimensional Arrays

• Test in each dimension separately.

• This can introduce another level of inaccuracy.

• Some tests (FM and Omega test) can test in many dimensions at the same time.

• Otherwise, you can linearize an array: Transform a logically N-dimensional array to its one-dimensional storage format.

Page 68: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 68

Hierarchy of Tests

• Try cheap test, then more expensive ones:• if (cheap test1= NO)• then print ‘NO’• else if (test2 = NO)• then print ‘NO’• else if (test3 = NO)• then print ‘NO’• else

Page 69: HC TD51021 Loop Transformations Motivation Loop level transformations catalogus –Loop merging –Loop interchange –Loop unrolling –Unroll-and-Jam –Loop tiling

HC TD5102 69

Practical Dependence Testing

• Cheap tests, like GCD and Banerjee tests, can disprove many dependences.

• Adding expensive tests only disproves a few more possible dependences.

• Compiler writer needs to trade-off compilation time and accuracy of dependence testing.

• For time critical applications, expensive tests like Omega test (exact!) can be used.


Heterogeneous Multicore Parallel Programming · 2009. 10. 6. · 3. Shared memory o Exploit data reuse (temporal data locality) 4. Register exploitation o Unroll (and Jam) 27 Loop

Carnegie Mellon Lecture 15 Loop Transformations Chapter 11.10-11.11 Dror E. MaydanCS243: Loop Optimization and Array Analysis1

Improving Software Pipelining with Unroll-and-Jam and Memory Reuse

Stanford University CS243 Winter 2006 Wei Li 1 Loop Transformations and Locality

Document No. FL-600-01 - usp.br · 8 Loop Transformations.....135 Scalar Replacement (IA-32 Only).....135

Lab: Introduction to Loop Transformations Tomofumi Yuki EJCP 2015 June 22, Nancy

High-Level Loop Transformations and Polyhedral Compilation

Embedded Computer Architecture TU/e 5kk73 Henk Corporaal Bart Mesman Loop Transformations

Lecture 15 Loop Transformations

How to use unroll me

Chapter 9 Working with the Loop Transformations ·  · 2014-04-26Chapter 9 Working with the Loop Transformations 9.1 Introduction to the Loop Transformations ... 17 Steps for an

Data-centric Transformations for Locality Enhancementment statements are contained in the innermost loop of the loop nest. The matrix multiplication code shown in Figure 1(e) is an

ELEMENTARY MATH - Home · Web viewChallenging iPad Apps Unroll Me Logic Puzzles

PREDICTING UNROLL FACTORS USING SUPERVISED LEARNING

Putting Polyhedral Loop Transformations to Work … Polyhedral Loop Transformations to Work Unified Model and Compiler Interface Albert Cohen with Cédric Bastoul, Sylvain Girbal,

Efficient Pipelining of Nested Loops: Unroll-and-Squash · Efficient Pipelining of Nested Loops: Unroll-and-Squash by ... Submitted to the Department of Electrical Engineering and

Locality-enhancing loop transformations for parallel tree traversal

efficient automatic verification of loop and data-flow transformations by functional equivalence

Locality-enhancing loop transformations for parallel tree

May 4, 2015 Loop Transformations 1 - University of Texas ...lin/cs380c/handout28.pdf · Calvin Lin, The University of Texas at Austin CS380 C Compilers 3 May 4, 2015 Loop Transformations

Loop Transformations - Colorado State Universitycs553/ClassNotes/lecture10-loop-dependencie… · – Midterm is Wednesday next week, ... – Intro to loop transformations ... Loop

Loop Transformations and Locality

1. Les transformations physiques Les transformations chimiques Les transformations nucléaires Les transformations physiques Les transformations chimiques

NETWORK ANALYSIS (15EC34) Syllabus:- · Syllabus:-Module -1 Basic Concepts: Practical sources, Source transformations, Network reduction using Star – Delta transformation, Loop

Revisiting Loop Transformations with X10 Clocks Tomofumi Yuki Inria / LIP / ENS Lyon X10 Workshop 2015

Lecture: Static ILPrajeev/cs6810/pres/20-6810-09.pdf · 3 Loop Unrolling • Increases program size • Requires more registers • To unroll an n-iteration loop by degree k, we will

Chapter 9 Working with the Loop Transformations

Cache-oblivious Programming. Story so far We have studied cache optimizations for array programs –Main transformations: loop interchange, loop tiling

Transformations - content.sandbox-njctl.orgcontent.sandbox-njctl.org/courses/math/geometry/transformations/... · Geometry – Transformations ~1~ NJCTL.org Transformations Transformations:

ECE 1754 Loop Transformations by: Eric LaForest. Motivation Improving loop behaviour/performance usually parallelism sometimes memory, register usage

Loop Transformations

Instruction Rescheduling and Loop-Unroll Department of Computer Science Southern Illinois University Edwardsville Fall, 2015 Dr. Hiroshi Fujinoki E-mail:

Dependence Analysis and Loop Transformations

Embedded Systems in Silicon TD5102 Data Management (2) Loop transformations & Data reuse Henk Corporaal heco/courses/EmbSystems

Efficient Pipelining of Nested Loops: Unroll-and-Squashgroups.csail.mit.edu/commit/papers/01/petkov-MEthesis.pdf · 2009-01-29 · Efficient Pipelining of Nested Loops: Unroll-and-Squash