the 5th international conference on automatic differentiation, bonn, germany, august 15, 2008
DESCRIPTION
Design and Implementation of a Context-Sensitive, Flow-Sensitive Activity Analysis Algorithm for Automatic Differentiation. The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008. Outline. Activity Analysis Previous work: CSFI New CSFS Algorithm - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/1.jpg)
Jaewook Shin, Priyadarshini Malusare and Paul D. HovlandMathematics and Computer Science Division
Argonne National Laboratory
Design and Implementation of a Context-Sensitive,Flow-Sensitive Activity Analysis Algorithm for
Automatic Differentiation
The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008
![Page 2: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/2.jpg)
2
Outline
1. Activity Analysis2. Previous work: CSFI3. New CSFS Algorithm4. Experimental Results5. Conclusion
![Page 3: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/3.jpg)
3
Activity Analysis
AD is applied to a function with a set of input variables and a set of output variables.
Sometimes, we are interested in the derivatives – of a subset of the output variables dependent – with respect to a subset of the input variables independent
An intermediate variable is – varied if it is transitively dependent on any independent variable – useful if any dependent variable is transitively dependent on it– active if it is both varied and useful.
Partial derivatives need to be computed only for active variables.
Activity analysis is nonseparable.
![Page 4: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/4.jpg)
4
Activity Analysis: Given f, compute dy1/dx1
f(x1, x2, x3, y1, y2, y3){ t1 = sin(x1) y1 = t1*2 t2 = cos(x2) y2 = t2*3 y3 = x3*t1}
f(x1, x2, x3, y1, y2, y3){ t1 = sin(x1) dt1/dx1 = cos(x1) y1 = t1*2 dy1/dt1 = 2 t2 = cos(x2) y2 = t2*3 y3 = x3*t1 dy1/dx1 = (dy1/dt1)*(dt1/dx1)}
![Page 5: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/5.jpg)
5
Activity Analysis: Given f, compute dy1/dx1
f(x1, x2, x3, y1, y2, y3){ t1 = sin(x1) y1 = t1*2 t2 = cos(x2) y2 = t2*3 y3 = x3*t1}
f(x1, x2, x3, y1, y2, y3){ t1 = sin(x1) dt1/dx1 = cos(x1) y1 = t1*2 dy1/dt1 = 2 t2 = cos(x2) dt2/dx2 = -sin(x2) y2 = t2*3 dy2/dt2 = 3 y3 = x3*t1 dy3/dx3 = t1 dy3/dt1 = x3 dy1/dx1 = (dy1/dt1)*(dt1/dx1) dy2/dx2 = (dt2/dx2)*(dy2/dt2) dy3/dx1 = (dt1/dx1)*(dy3/dt1)}
![Page 6: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/6.jpg)
6
Activity Analysis: varied + useful = active
f(x1, x2, x3, y1, y2, y3){ t1 = sin(x1); y1 = t1*2; t2 = cos(x2); y2 = t2*3; y3 = x3*t1;}
f(x1, x2, x3, y1, y2, y3){ t1 = sin(x1); y1 = t1*2; t2 = cos(x2); y2 = t2*3; y3 = x3*t1;}
f(x1, x2, x3, y1, y2, y3){ t1 = sin(x1); y1 = t1*2; t2 = cos(x2); y2 = t2*3; y3 = x3*t1;}
f(x1, x2, x3, y1, y2, y3){ t1 = sin(x1); y1 = t1*2; t2 = cos(x2); y2 = t2*3; y3 = x3*t1;}
varied
useful
varied
usefulactive
![Page 7: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/7.jpg)
7
Previous work: Context-Sensitive, Flow-Insensitive Activity Analysis (VDGAA)
Graph reachability problem Variable Dependence Graph (VDG) Two separate (color) propagations:
– Forward (coloring red) for “varied” variables – Backward (coloring yellow) for “useful” variables
Context sensitivity is supported by a stack of contexts.
Run time– Very fast in practice
Small overestimations of active variables due to– the way programs are usually written– the property of activity analysis
![Page 8: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/8.jpg)
8
VDGAA on the run: 1. build VDGAA
f(x1, x2, x3, y1, y2, y3){ t1 = sin(x1); y1 = t1*2; t2 = cos(x2); y2 = t2*3; y3 = x3*t1;}
x1
x2
x3
t2
y1
y2
y3t1
Variable Dependence Graph (VDG)
![Page 9: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/9.jpg)
9
VDGAA on the run: 2. Find ‘varied’ variables
f(x1, x2, x3, y1, y2, y3){ t1 = sin(x1); y1 = t1*2; t2 = cos(x2); y2 = t2*3; y3 = x3*t1;}
x1
x2
x3
t2
y1
y2
y3t1Forward propagation
![Page 10: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/10.jpg)
10
VDGAA on the run: 3. Find ‘useful’ variables
f(x1, x2, x3, y1, y2, y3){ t1 = sin(x1); y1 = t1*2; t2 = cos(x2); y2 = t2*3; y3 = x3*t1;}
x1
x2
x3
t2
y1
y2
y3t1Backward propagation
![Page 11: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/11.jpg)
11
VDGAA on the run: Context Sensitivity
foo(a, b){ b = a;}
x p1 p2 y
ba
Forward propagation
call foo(x, p1); call foo(p2, y);
![Page 12: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/12.jpg)
12
VDGAA on the run: Context Sensitivity
foo(a, b){ b = a;}
x p1 p2 y
ba
Backward propagation
call foo(x, p1); call foo(p2, y);
![Page 13: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/13.jpg)
13
f(x1, x2, x3, y1, y2, y3){ t1 = sin(x1); y1 = t1*2; y3 = x3*t1; t1 = cos(x2); y2 = t1*3;}
VDGAA : Flow Insensitivity
x1
x2
x3
y1
y2
y3t1
Forward propagation
![Page 14: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/14.jpg)
14
VDGAA : Flow Insensitivity
f(x1, x2, x3, y1, y2, y3){ t1 = sin(x1); y1 = t1*2; y3 = x3*t1; t1 = cos(x2); y2 = t1*3;}
x1
x2
x3
y1
y2
y3t1
Backward propagation
![Page 15: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/15.jpg)
15
Context-Sensitive, Flow-Sensitive Activity Analysis (DUGAA)
Graph reachability problem Two separate (color) propagations:
– Forward for “varied” variables– Backward for “useful” variables
Context sensitivity is supported by a stack of contexts.
Definition-Use Graph (DUG) Flow sensitivity is supported by the use of reaching definitions for nodes.
Algorithm:
UD-DUChains
Build adef-use graph
Forwardpropagation
Backwardpropagation
![Page 16: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/16.jpg)
16
DUGAA on the run: 1. build a DUG
1:f(x1, x2, x3, y1, y2, y3){2: t1 = sin(x1);3: y1 = t1*2;4: y3 = x3*t1;5: t1 = cos(x2);6: y2 = t1*3;7:}
x1@I
x2@I
x3@I
y1@O
y2@O
y3@O
t1@2
t1@5
y1@3
y3@4y2@6
Def-Use Graph (DUG)
![Page 17: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/17.jpg)
17
DUGAA on the run: 2. Forward propagation
1:f(x1, x2, x3, y1, y2, y3){2: t1 = sin(x1);3: y1 = t1*2;4: y3 = x3*t1;5: t1 = cos(x2);6: y2 = t1*3;7:}
x1@I
x2@I
x3@I
y1@O
y2@O
y3@O
t1@2
t1@5
y1@3
y3@4y2@6
Forward propagation
![Page 18: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/18.jpg)
18
DUGAA on the run: 3. Backward propagation
1:f(x1, x2, x3, y1, y2, y3){2: t1 = sin(x1);3: y1 = t1*2;4: y3 = x3*t1;5: t1 = cos(x2);6: y2 = t1*3;7:}
x1@I
x2@I
x3@I
y1@O
y2@O
y3@O
t1@2
t1@5
y1@3
y3@4y2@6
Backward propagation
![Page 19: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/19.jpg)
19
Implementation
OpenAnalysis
VDGAA
DUGAA
OpenAD: ADTransformation
Open64Unparser
Open64Front end
Input(Fortran)
Output(Fortran)
![Page 20: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/20.jpg)
20
Benchmarks
Benchmarks Description Source #lines
MITgcm MIT General Circulation Model MIT 27376
LU Lower-upper symmetric Gauss-Seidel NASPB 5951
CG Conjugate gradient NASPB 2480
newton Newton’s method + Rosenbrock function ANL 2189
adiabatic Adiabatic flow model in chemical engineering CMU 1009
msa Minimal surface area problem MINPACK-2 461
swirl Swirling flow problem MINPACK-2 355
c2 Ordinary differential equation solver ANL 64
![Page 21: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/21.jpg)
21
Slowdowns in analysis run time: DUGAA vs. VDGAA
30.89
105.76
66.5
inf
31.67 27
74
inf
0
30
60
90
120
150
MIT gcm
LU CG newton
adiabatic
msaswirl
c2
![Page 22: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/22.jpg)
22
Analysis run time
1.71
52.82
0.17
17.98
0.02
1.33
0.0
1.27
0.03
0.95
0.01
0.27
0.01
0.74
0.00.010
0.5
1
1.5
2(Seconds)
VDGAA DUGAA
![Page 23: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/23.jpg)
23
Analysis run-time breakdown on MITgcm
0
81.26
35.09
1.72
53.8
16.86 11.110.660
20406080
100(% of run time)
UD-DU Chains
Graph generation
Transitive closure
Coloring
VDGAA DUGAA
![Page 24: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/24.jpg)
24
Reduction in active variables
8/925
0/311 0/12 0/2 0/141 0/13 0/4
1/6
0
2
4
6
8
MIT gcm
LU CG newton
adiabatic
msaswirl
c2
![Page 25: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/25.jpg)
25
Conclusion
A new context-sensitive, flow-sensitive (CSFS) activity analysis algorithm: Def-Use Graph Activity Analysis (DUGAA)
Comparison of two activity analyses: DUGAA (CSFS) vs. VDGAA (CSFI)
Slower than VDGAA for all 8 benchmarks – by a factor > 27– but takes less than one minute for a code larger than 27k lines
Makes fewer overestimations than VDGAA for two of the eight benchmarks. May save human effort in managing AD code.
Future work– Comparison among CIFS, CSFI, and CSFS activity analyses– Dealing with pointers and recursion
![Page 26: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/26.jpg)
26
PARAM Edges
Edges between formal parameters Summarize the connectivity among formal parameters Transitive closure is applied to the dependence matrix of local variables
Checking for connectivity through global variables: A PARAM edge is generated from formal variable node F1 to formal variable node F2 by traversing the entire graph only when– F1 has a value flow path to a global variable node AND– F2 has a value flow path from a global variable node.
![Page 27: The 5th International Conference on Automatic Differentiation, Bonn, Germany, August 15, 2008](https://reader036.vdocuments.mx/reader036/viewer/2022062500/56815aa3550346895dc82b37/html5/thumbnails/27.jpg)
27
Context-Insensitive, Flow-Sensitive Activity Analysis (ICFGAA)
Interprocedural Control Flow Graph (ICFG) Iterative data-flow analysis (DFA) Two separate DFAs:
– Forward for “varied” variables– Backward for “useful” variables
Long run time– Large number of iterations– Nonseparability of activity analysis: data-flow analysis values depend
on other data-flow values Large overestimation
– Due to context-insensitivity– Value propagation through unrealizable control paths