program analysis via graph reachability
DESCRIPTION
Program Analysis via Graph Reachability. Thomas Reps University of Wisconsin. http://www.cs.wisc.edu/~reps/. See http://www.cs.wisc.edu/wpis/papers/tr1386.ps. PLDI 00 Registration Form. PLDI 00: …………………….. $ ____ Tutorial (morning): …………… $ ____ Tutorial (afternoon): ………….. $ ____ - PowerPoint PPT PresentationTRANSCRIPT
Program Analysis via Graph Reachability
Thomas Reps
University of Wisconsinhttp://www.cs.wisc.edu/~reps/
See http://www.cs.wisc.edu/wpis/papers/tr1386.ps
PLDI 00 Registration Form
• PLDI 00: …………………….. $ ____
• Tutorial (morning): …………… $ ____
• Tutorial (afternoon): ………….. $ ____
• Tutorial (evening): ……………. $ – 0 –
1987
1993
1994
1995
1997
1998
1996
Slicing&
Applications
DataflowAnalysis Demand
Algorithms
SetConstraints
Structure-TransmittedDependences
CFLReachability
Applications
• Program optimization
• Software engineering– Program understanding
– Reengineering
– Static bug-finding
• Security (information flow)
Collaborators
• Susan Horwitz
• Mooly Sagiv
• Genevieve Rosay
• David Melski
• David Binkley
• Michael Benedikt
• Patrice Godefroid
Themes
• Harnessing CFL-reachability
• Exhaustive alg. Demand alg.
int main() {int sum = 0;int i = 1;while (i < 11) {
sum = sum + i;i = i + 1;
}printf(“%d\n”,sum);printf(“%d\n”,i);
}
Backward Slice
Backward slice with respect to “printf(“%d\n”,i)”
int main() {int sum = 0;int i = 1;while (i < 11) {
sum = sum + i;i = i + 1;
}printf(“%d\n”,sum);printf(“%d\n”,i);
}
Backward Slice
Backward slice with respect to “printf(“%d\n”,i)”
int main() {
int i = 1;while (i < 11) {
i = i + 1;}
printf(“%d\n”,i);}
Slice Extraction
Backward slice with respect to “printf(“%d\n”,i)”
Forward Slice
int main() {int sum = 0;int i = 1;while (i < 11) {
sum = sum + i;i = i + 1;
}printf(“%d\n”,sum);printf(“%d\n”,i);
}
Forward slice with respect to “sum = 0”
Forward slice with respect to “sum = 0”
Forward Slice
int main() {int sum = 0;int i = 1;while (i < 11) {
sum = sum + i;i = i + 1;
}printf(“%d\n”,sum);printf(“%d\n”,i);
}
What Are Slices Useful For?• Understanding Programs
– What is affected by what?
• Restructuring Programs– Isolation of separate “computational threads”
• Program Specialization and Reuse– Slices = specialized programs– Only reuse needed slices
• Program Differencing– Compare slices to identify changes
• Testing– What new test cases would improve coverage?– What regression tests must be rerun after a change?
Line-Character-Count Program
void line_char_count(FILE *f) {int lines = 0;int chars;BOOL eof_flag = FALSE;int n;extern void scan_line(FILE *f, BOOL *bptr, int *iptr);scan_line(f, &eof_flag, &n);chars = n;while(eof_flag == FALSE){
lines = lines + 1;scan_line(f, &eof_flag, &n);chars = chars + n;
}printf(“lines = %d\n”, lines);printf(“chars = %d\n”, chars);
}
Character-Count Program
void char_count(FILE *f) {int lines = 0;int chars;BOOL eof_flag = FALSE;int n;extern void scan_line(FILE *f, BOOL *bptr, int *iptr);scan_line(f, &eof_flag, &n);chars = n;while(eof_flag == FALSE){
lines = lines + 1;scan_line(f, &eof_flag, &n);chars = chars + n;
}printf(“lines = %d\n”, lines);printf(“chars = %d\n”, chars);
}
Line-Character-Count Program
void line_char_count(FILE *f) {int lines = 0;int chars;BOOL eof_flag = FALSE;int n;extern void scan_line(FILE *f, BOOL *bptr, int *iptr);scan_line(f, &eof_flag, &n);chars = n;while(eof_flag == FALSE){
lines = lines + 1;scan_line(f, &eof_flag, &n);chars = chars + n;
}printf(“lines = %d\n”, lines);printf(“chars = %d\n”, chars);
}
Line-Count Program
void line_count(FILE *f) {int lines = 0;int chars;BOOL eof_flag = FALSE;int n;extern void scan_line2(FILE *f, BOOL *bptr, int *iptr);scan_line2(f, &eof_flag, &n);chars = n;while(eof_flag == FALSE){
lines = lines + 1;scan_line2(f, &eof_flag, &n);chars = chars + n;
}printf(“lines = %d\n”, lines);printf(“chars = %d\n”, chars);
}
Specialization Via Slicing
wc -lc
wc -c wc -l
Control Flow Graph
Enter
sum = 0 i = 1 while(i < 11) printf(sum) printf(i)
sum = sum + i i = i + i
T
F
int main() {int sum = 0;int i = 1;while (i < 11) {
sum = sum + i;i = i + 1;
}printf(“%d\n”,sum);printf(“%d\n”,i);
}
q is reached from pif condition p istrue (T), not otherwise.
Control Dependence Graph
Control dependence
p qT
p qF
Similar for false (F).
Enter
sum = 0 i = 1 while(i < 11) printf(sum) printf(i)
sum = sum + i i = i + i
T T
TT T
TTT
int main() {int sum = 0;int i = 1;while (i < 11) {
sum = sum + i;i = i + 1;
}printf(“%d\n”,sum);printf(“%d\n”,i);
}
Flow Dependence Graphint main() {
int sum = 0;int i = 1;while (i < 11) {
sum = sum + i;i = i + 1;
}printf(“%d\n”,sum);printf(“%d\n”,i);
} Enter
sum = 0 printf(sum) printf(i)
sum = sum + i i = i + i
Flow dependence
p q Value of variableassigned at p may beused at q.
i = 1 while(i < 11)
Program Dependence Graph (PDG)int main() {
int sum = 0;int i = 1;while (i < 11) {
sum = sum + i;i = i + 1;
}printf(“%d\n”,sum);printf(“%d\n”,i);
} Enter
sum = 0 i = 1 while(i < 11) printf(sum) printf(i)
sum = sum + i i = i + i
T
TT T
T
Control dependence
Flow dependence
TT
T
Program Dependence Graph (PDG)int main() {
int i = 1;int sum = 0;while (i < 11) {
sum = sum + i;i = i + 1;
}printf(“%d\n”,sum);printf(“%d\n”,i);
} Enter
sum = 0 i = 1 while(i < 11) printf(sum) printf(i)
sum = sum + i i = i + i
T
TT T
TTT
T
Opposite Order
Same PDG
Backward Sliceint main() {
int sum = 0;int i = 1;while (i < 11) {
sum = sum + i;i = i + 1;
}printf(“%d\n”,sum);printf(“%d\n”,i);
} Enter
sum = 0 i = 1 while(i < 11) printf(sum) printf(i)
sum = sum + i i = i + i
T
TT T
TTT
T
Backward Slice (2)int main() {
int sum = 0;int i = 1;while (i < 11) {
sum = sum + i;i = i + 1;
}printf(“%d\n”,sum);printf(“%d\n”,i);
} Enter
sum = 0 i = 1 while(i < 11) printf(sum) printf(i)
sum = sum + i i = i + i
T
TT T
TTT
T
Backward Slice (3)int main() {
int sum = 0;int i = 1;while (i < 11) {
sum = sum + i;i = i + 1;
}printf(“%d\n”,sum);printf(“%d\n”,i);
} Enter
sum = 0 i = 1 while(i < 11) printf(sum) printf(i)
sum = sum + i i = i + i
T
TT T
TTT
T
Backward Slice (4)int main() {
int sum = 0;int i = 1;while (i < 11) {
sum = sum + i;i = i + 1;
}printf(“%d\n”,sum);printf(“%d\n”,i);
} Enter
sum = 0 i = 1 while(i < 11) printf(sum) printf(i)
sum = sum + i i = i + i
TT
TT T
TTT
Slice Extractionint main() {
int i = 1;while (i < 11) {
i = i + 1;}
printf(“%d\n”,i);} Enter
i = 1 while(i < 11) printf(i)
i = i + iT
TT
TT
CodeSurfer
Browsing a Dependence Graph
Pretend this is your favorite browser
What does clicking on a link do?You geta new page
Or you move to an internal tag
Interprocedural Slice
int main() {int sum = 0;int i = 1;while (i < 11) {
sum = add(sum,i);i = add(i,1);
}printf(“%d\n”,sum);printf(“%d\n”,i);
}
int add(int x, int y) {return x + y;
}
Backward slice with respect to “printf(“%d\n”,i)”
Interprocedural Slice
int main() {int sum = 0;int i = 1;while (i < 11) {
sum = add(sum,i);i = add(i,1);
}printf(“%d\n”,sum);printf(“%d\n”,i);
}
int add(int x, int y) {return x + y;
}
Backward slice with respect to “printf(“%d\n”,i)”
int main() {int sum = 0;int i = 1;while (i < 11) {
sum = add(sum,i);i = add(i,1);
}printf(“%d\n”,sum);printf(“%d\n”,i);
}
Interprocedural Slice
int add(int x, int y) {return x + y;
}
Superfluous components included by Weiser’s slicing algorithm [TSE 84]Left out by algorithm of Horwitz, Reps, & Binkley [PLDI 88; TOPLAS 90]
System Dependence Graph (SDG)
Enter main
Call p Call p
Enter p
SDG for the Sum ProgramEnter main
sum = 0 i = 1 while(i < 11) printf(sum) printf(i)
Call add Call add
xin = sum yin = i sum = xout xin = i yin= 1 i = xout
Enter add
x = xin y = yin x = x + y xout = x
Interprocedural Backward SliceEnter main
Call p Call p
Enter p
Interprocedural Backward Slice (2)Enter main
Call p Call p
Enter p
Interprocedural Backward Slice (3)Enter main
Call p Call p
Enter p
Interprocedural Backward Slice (4)Enter main
Call p Call p
Enter p
Interprocedural Backward Slice (5)Enter main
Call p Call p
Enter p
Interprocedural Backward Slice (6)Enter main
Call p Call p
Enter p
[
]
)
(
Matched-Parenthesis Path
)(
)[
Interprocedural Backward Slice (6)Enter main
Call p Call p
Enter p
Interprocedural Backward Slice (7)Enter main
Call p Call p
Enter p
Slice ExtractionEnter main
Call p
Enter p
Slice of the Sum ProgramEnter main
i = 1 while(i < 11) printf(i)
Call add
xin = i yin= 1 i = xout
Enter add
x = xin y = yin x = x + y xout = x
CFL-Reachability[Yannakakis 90]
• G: Graph (N nodes, E edges)
• L: A context-free language
• L-path from s to t iff
• Running time: O(N 3)
Lts ,*
Interprocedural Slicingvia CFL-Reachability
• Graph: System dependence graph
• L: L(matched) [roughly]
• Node m is in the slice w.r.t. n iff there
is an L(matched)-path from m to n
Asymptotic Running Time [Reps, Horwitz, Sagiv, & Rosay 94]
• CFL-reachability
– System dependence graph: N nodes, E edges
– Running time: O(N 3)
• System dependence graph Special structure
Running time: O(E + CallSites MaxParams3)
( e [
e
]e
[
e ] ] e )
matched | e | [ matched ] | ( matched ) | matched matched
CFL-Reachability
s ts
( e e e e e e[[ [
t
)]] ]
s ts t
Ordinary Graph Reachability
Regular-Language Reachability[Yannakakis 90]
• G: Graph (N nodes, E edges)
• L: A regular language
• L-path from s to t iff
• Running time: O(N+E)
• Ordinary reachability (= transitive closure)
– Label each edge with e
– L is e*
Lts ,*
vs. O(N3)
CFL-Reachability via Dynamic Programming
GrammarGraph
BC
A
A B C
s t
Degenerate Case: CFL-Recognition
“(a + b) * c” L(exp) ?
exp id | exp + exp | exp * exp | ( exp )
)( a cb+ *
*a + +)b c
s t
Degenerate Case: CFL-Recognition
“a + b) * c +” L(exp) ?
exp id | exp + exp | exp * exp | ( exp )
S T
Program Chopping
Given source S and target T, what program points transmit effects from S to T?
Intersect forward slice from S with backward slice from T, right?
Non-Transitivity and Slicing
int main() {int sum = 0;int i = 1;while (i < 11) {
sum = add(sum,i);i = add(i,1);
}printf(“%d\n”,sum);printf(“%d\n”,i);
}
int add(int x, int y) {return x + y;
}
Forward slice with respect to “sum = 0”
int main() {int sum = 0;int i = 1;while (i < 11) {
sum = add(sum,i);i = add(i,1);
}printf(“%d\n”,sum);printf(“%d\n”,i);
}
Forward slice with respect to “sum = 0”
Non-Transitivity and Slicing
int add(int x, int y) {return x + y;
}
Non-Transitivity and Slicing
int main() {int sum = 0;int i = 1;while (i < 11) {
sum = add(sum,i);i = add(i,1);
}printf(“%d\n”,sum);printf(“%d\n”,i);
}
int add(int x, int y) {return x + y;
}
Backward slice with respect to “printf(“%d\n”,i)”
Non-Transitivity and Slicing
int main() {int sum = 0;int i = 1;while (i < 11) {
sum = add(sum,i);i = add(i,1);
}printf(“%d\n”,sum);printf(“%d\n”,i);
}
int add(int x, int y) {return x + y;
}
Backward slice with respect to “printf(“%d\n”,i)”
Forward slice with respect to “sum = 0”
Non-Transitivity and Slicing
int main() {int sum = 0;int i = 1;while (i < 11) {
sum = add(sum,i);i = add(i,1);
}printf(“%d\n”,sum);printf(“%d\n”,i);
}
int add(int x, int y) {return x + y;
}
Backward slice with respect to “printf(“%d\n”,i)”
Non-Transitivity and Slicing
int main() {int sum = 0;int i = 1;while (i < 11) {
sum = add(sum,i);i = add(i,1);
}printf(“%d\n”,sum);printf(“%d\n”,i);
}
int add(int x, int y) {return x + y;
}
Chop with respect to “sum = 0” and “printf(“%d\n”,i)”
Non-Transitivity and SlicingEnter main
sum = 0 i = 1 while(i < 11) printf(sum) printf(i)
Call add Call add
xin = sum yin = i sum = xout xin = i yin= 1 i = xout
Enter add
x = xin y = yin x = x + y xout = x
( ]
Program Chopping
Given source S and target T, what program points transmit effects from S to T?
S T
“Precise interprocedural chopping”[Reps & Rosay FSE 95]
Dataflow Analysis
• Goal: For each point in the program, determine a superset of the “facts” that could possibly hold during execution
• Examples– Constant propagation– Reaching definitions– Live variables– Possibly uninitialized variables
Possibly Uninitialized VariablesStart
x = 3
if . . .
y = x
y = w
w = 8
printf(y)
},,.{ yxwV
}{. xVV
VV .VV .
}{. wVV
}{ else }{ then
if .
yVyV
VxV
}{ else }{ then
if .
yVyV
VwV
{w,x,y}
{w,y}
{w,y}
{w,y}
{w}
{w,y}{}
{w,y}
{}
Precise Intraprocedural Analysis
start n
C f 1 f 2 f kf 1k
pfp = fk fk-1 … f2 f1
MOP[n] = pfp(C) pPathsTo[n]
x = 3
p(x,y)
return from p
printf(y)
start main
exit main
start p(a,b)
if . . .
b = a
p(a,b)
return from p
printf(b)
exit p
(
)
]
(
Precise Interprocedural Analysis
start n
C
f 4
f 5
f 3
start q exitq
callq retf 1 f 2 f kf 1k
f 2k
f 3k
( )
[Sharir & Pnueli 81]
MOMP[n] = pfp(C) pMatchedPathsTo[n]
Representing Dataflow Functions
Identity Function
VV .f
}{.f bVConstant Function
a b c
a b c
},{}),f({ baba
}{}),f({ bba
Representing Dataflow Functions
}{}){.(f cbVV
}{ else }{ then
if .f
bVbV
VaV
“Gen/Kill” Function
Non-“Gen/Kill” Function a b c
a b c
},{}),f({ caba
},{}),f({ baba
x = 3
p(x,y)
return from p
printf(y)
start main
exit main
start p(a,b)
if . . .
b = a
p(a,b)
return from p
printf(b)
exit p
x y a b
else }{ then
if .f 2
cVbV
a b c}{ else }{then
if .f 1
bVbV
VaV
b ca
Composing Dataflow Functions
}{ else }{then
if .f 1
bVbV
VaV
b ca
else }{ then
if .f 2
cVbV
}{cf2 f1({a,c}) =
x = 3
p(x,y)
return from p
start main
exit main
start p(a,b)
if . . .
b = a
p(a,b)
return from p
exit p
x y a b
printf(y)
Might b beuninitializedhere?
printf(b) NO!
(
]
Might y beuninitializedhere?
YES!
(
)
matched matched matched
| (i matched )i 1 i CallSites | edge |
stack
) ( (
((
(
)
) )
)
stack
( )
Off Limits!
) (
(
((
(
)
)
)
( )
(
stack
(
(
unbalLeft matched unbalLeft
| (i unbalLeft 1 i CallSites |
stack
Off Limits!
Interprocedural Dataflow Analysisvia CFL-Reachability
• Graph: Exploded control-flow graph
• L: L(unbalLeft)
• Fact d holds at n iff there is an L(unbalLeft)-path
from dnstartmain , to,
Asymptotic Running Time [Reps, Horwitz, & Sagiv 95]
• CFL-reachability– Exploded control-flow graph: ND nodes– Running time: O(N3D3)
• Exploded control-flow graph Special structure
Running time: O(ED3)
Typically: E N, hence O(ED3) O(ND3)
“Gen/kill” problems: O(ED)
Why Bother?“We’re only interested in million-line programs”
• Know thy enemy!– “Any” algorithm must do these operations– Avoid pitfalls (e.g., claiming O(N2) algorithm)
• The essence of “context sensitivity”• Special cases
– “Gen/kill” problems: O(ED)• Compression techniques
– Basic blocks– SSA form, sparse evaluation graphs
• Demand algorithms
Unifying Conceptual Modelfor Dataflow-Analysis Literature
• Linear-time gen-kill [Hecht 76], [Kou 77]• Path-constrained DFA [Holley & Rosen 81]• Linear-time GMOD [Cooper & Kennedy 88]• Flow-sensitive MOD [Callahan 88]• Linear-time interprocedural gen-kill
[Knoop & Steffen 93]• Linear-time bidirectional gen-kill [Dhamdhere 94]• Relationship to interprocedural DFA
[Sharir & Pneuli 81], [Knoop & Steffen 92]
Themes
• Harnessing CFL-reachability
• Exhaustive alg. Demand alg.
Exhaustive Versus Demand Analysis
• Exhaustive analysis: All facts at all points
• Optimization: Concentrate on inner loops
• Program-understanding tools: Only some facts
are of interest
Exhaustive Versus Demand Analysis
• Demand analysis:– Does a given fact hold at a given point?– Which facts hold at a given point?– At which points does a given fact hold?
• Demand analysis via CFL-reachability– single-source/single-target CFL-reachability– single-source/multi-target CFL-reachability– multi-source/single-target CFL-reachability
x = 3
p(x,y)
return from p
printf(y)
start main
exit main
start p(a,b)
if . . .
b = a
p(a,b)
return from p
printf(b)
exit p
x y a b
YES!
(
)
NO!
“Semi-exhaustive”:All “appropriate” demands
Might y beuninitializedhere?
Might b beuninitializedhere?
Experimental Results[Horwitz , Reps, & Sagiv 1995]
• 53 C programs (200-6,700 lines)• For a single fact of interest:
– demand always better than exhaustive
• All “appropriate” demands beats exhaustive when percentage of “yes” answers is high– Live variables– Truly live variables– Constant predicates– . . .
A Related Result [Sagiv, Reps, & Horwitz 1996]
• [Uses a generalized analysis technique]• 38 C programs (300-6,000 lines)
– copy-constant propagation– linear-constant propagation
• All “appropriate” demands always beats exhaustive– factor of 1.14 to about 6
Exhaustive Versus Demand Analysis
• Demand algorithms for
– Interprocedural dataflow analysis
– Set constraints
– Points-to analysis
Most Significant Contributions: 1987-2000
• Asymptotically fastest algorithms– Interprocedural slicing– Interprocedural dataflow analysis
• Demand algorithms– Interprocedural dataflow analysis [CC94,FSE95]– All “appropriate” demands beats exhaustive
• Tool for slicing and browsing ANSI C– Slices programs as large as 75,000 lines– University research distribution– Commercial product: CodeSurfer
(GrammaTech, Inc.)
References
• Papers by Reps and collaborators:– http://www.cs.wisc.edu/~reps/
• CFL-reachability– Yannakakis, M., Graph-theoretic methods in
database theory, PODS 90.– Reps, T., Program analysis via graph
reachability, Inf. and Softw. Tech. 98.
References
• Slicing, chopping, etc.– Horwitz, Reps, & Binkley, TOPLAS 90– Reps, Horwitz, Sagiv, & Rosay, FSE 94– Reps & Rosay, FSE 95
• Dataflow analysis– Reps, Horwitz, & Sagiv, POPL 95– Horwitz, Reps, & Sagiv, FSE 95, TR-1283