analysis of path profiling information generated with performance monitoring hardware
DESCRIPTION
Analysis of Path Profiling Information Generated with Performance Monitoring Hardware. Alex Shye, Matt Iyer, Tipp Moseley, Dave Hodgdon Dan Fay, Vijay Janapa Reddi, Dan Connors University of Colorado at Boulder Department of Electrical and Computer Engineering - PowerPoint PPT PresentationTRANSCRIPT
Analysis of Path Profiling Information Generated with
Performance Monitoring Hardware
Alex Shye, Matt Iyer, Tipp Moseley, Dave Hodgdon
Dan Fay, Vijay Janapa Reddi, Dan Connors
University of Colorado at Boulder
Department of Electrical and Computer Engineering
DRACO Architecture Research Group
Introduction
• Profile information is critical to success of optimizers
– Point Profile - BBs count, edge profiles, etc.– Path Profile - correlated branches
• Off-line Path Profiling Methods:– Use static/dynamic instrumentation to gather
full path profile[Ball96][Joshi04][Bond05]
• On-line Path Profiling Method:– Interpretation: MRET[Bala00][Bruen03]
• Both incur high overhead!!• For run-time systems, overhead
unacceptable
A
B C
D
E F
G
80 20
7030
Edge Profile: ABDFG 70-50
Path Profile: ABDFG 60 ACDFG 10 …
Performance Monitoring
• Modern processors contain on-chip Performance Monitoring Units(PMUs)– Itanium, Pentium 4, Power PC support branch vectors
• Sampling PMU– Less information– Non-deterministic, phase behavior
• Branch Execution Information– Itanium-2 PMU Branch Trace Buffer(BTB) - up to four branches
• Different configurations: Last-4 branches, Last-4 taken branches, etc
– Compiler can expand this information
PMU-based Path Profiling• Goal: Combine compiler analysis and PMU
branch vectors to generate a path profile
• In order for PMU-based path profiling to effective, it must to comparable to a full path profile ex. Ball Larus PP[Ball96]
• Other forms of PMU-based profile information have been shown to be effective at run-time optimization - ADORE[Chen03][Lu04]
Hot Path
BTB Trace
Hardware Profiling Approaches
• Proposed Techniques:– BTB profile buffer [Conte94]
• OS coupled with BTB hardware to fill out an edge profile– Hot Spot Detection[Merten00]
• Proposed Branch Behavior Buffer to store branch information to fill out edge profile
– Programmable Path Profiler [Vaswani05]
• Hardware Path Stack and Path Detector
• Performance Monitoring Unit Techniques– Continuous Profiling/Optimization Systems
• Simple PMU - event counters– ADORE Dynamic Optimizer [Chen03][Lu04]
• Sampling Itanium-2 PMU to drive memory optimizations
Motivation
• Unfortunately, most existing techniques are only able to accomplish one or two of these.
• This project aims to combine the accuracy of path profiling with low-overhead utilizing existing performance monitoring hardware.
Accuracy Single-Stage Low-Overhead
Static Instrumentation
Dynamic Instrumentation
Interpretation
Hardware Techniques
Characteristics of the Ideal Run-time Profiler
1. Accuracy - Ability to reflect run-time execution well
2. Single-Stage - Can profile binary on-the-fly without extra compilation stages
3. Low Overhead - Incurs little to no overhead
Itanium-2 PMU Path Profiling
• 2 Phases– Online
• BTB Trace Collection
– Offline• Partial Path Creation• Region Formation• Path Profile Generation
– Path Matching– Path Crediting
PMU
Processor
BTB Traces… Partial Paths
Compiler-Aided Offline Analysis
PATHS!Region
Formation
Path Matching/Crediting
TerminologyBTB Trace: Series of addresses from BTB
Partial Path: Path of ops in compiler IR
Region: Single Entrance region in CFG
Path: Complete path through a region
BTB Trace Collection
• BTB Trace: Sequence of four branches per sample– Configured to sample only taken branches
• Allows for longer partial paths to be built• The not taken path is trivial to follow
• BTB Trace placed into specialized hash table every sample– If BTB Trace exists, increment count
• At the end of execution, BTB Traces and counts are dumped to a file
Partial Path Creation
• Partial Path: List of low-level IR ops
• Partial Path Formation– Recreate path from BTB Trace– Partial Path weight = count– Perform Partial Path Extensions
• Up until Join Point• Down until Branch Point
Join Point
Branch Point
Partial Path from BTB Trace
Extended Partial Path
BTB Trace Branch
Path Matching and Crediting• Path Matching
– Find list of all paths that contain partial path
• Path Crediting– Distribute partial path weight equally among
matched paths
• Example:
• Challenge:– Number of paths grows exponentially– Large control flow graphs present a problem
A
CB
D
L
NM
O
E
GF
HQP
R
TS
U
WV
X
JI
K
Y
Partial Path Count Matches Inc Total
OPRSUVXY 100 ABDLMOPRSUVXY
ACDLMOPRSUVXY
ABDLNOPRSUVXY
ACDLNOPRSUVXY
+25
+25
+25
+25
25
25
25
25
Region 3
Region 2
Region 1
Region Formation• We use region-based paths
– Makes total # paths more manageable– Limits number of matching paths
• Rules for Region R:– R must be single entry– R may not cross loop boundaries
• Loop Regions created first
– R may not cross function boundaries– Total # paths in R is limited by a threshold– R must be as large as possible
• Side Effects of Region Formation– Partial Paths must be split at:
• Loop boundaries• Function boundaries• Region boundaries
A
CB
D
L
NM
O
E
GF
HQP
R
TS
U
WV
X
JI
K
Y
Path Generation Example• Suppose we encounter these paths:
– ABDLMOP– ABDEFHIK
• Split into ABD, EFHIK
– OPRSUVX
Partial Path Count Matches Inc Total
ABDLMOP 100 ABDLMOPRSUVX
ABDLMOPRSUWX
ABDLMOPRSUVX
ABDLMOPRSUWX
+25
+25
+25
+25
25
25
25
25
ABD 160 ABDLMOPRSUVX
…(14 more)
ABDLNOQRTUWX
+10
…
+10
35
10
EFHIK 160 EFHIK +160 160
OPRSUVX 280 ABDLMOPRSUVX
ABDLNOPRSUVX
ACDLMOPRSUVX
ACDLNOPRSUVX
+70
+70
+70
+70
105
80
70
70Region 3
Region 2
Region 1
A
CB
D
L
NM
O
E
GF
HQP
R
TS
U
WV
X
JI
K
Y
Methodology
• Experiments run on Itanium 2• Developed tool using perfmon kernel interface
and libpfm[perfmon] to interface with PMU• Benchmarks
– Set of SPEC2000 benchmarks– Compiled with the OpenIMPACT Research
Compiler[oicc]
• Without aggressive profile-directed optimizations
• Off-line analysis with OpenIMPACT module• Compared to full path profile gathered with a
PIN path profiling tool
Effect of Sampling Period
• Knee of Overhead curve ~500K• Number of Unique Paths consistently grows as sampling period decreases
– Levels off some between 50K and 100K
164.gzip
0
500
1000
1500
2000
50K 100K 500K 1M 5M 10M
Sampling Period
Unique Paths0
10
20
30
40
50
Overhead (%)
300.twolf
0
2000
4000
6000
8000
10000
50K 100K 500K 1M 5M 10M
Sampling Period
Unique Paths0102030405060
Overhead (%)
Accuracy Results• Accuracy measured similar to Wall’s weight matching scheme[Wall91]
Accuracy Vs. Sampling Period
0
10
20
30
40
50
60
70
80
90
100
50K 100K500K1M 5M 10M 50M100M500M
Sampling Period
Accuracy (%)
164.gzip175.vpr177.mesa179.art181.mcf183.equake188.ammp197.parser256.bzip2300.twolf
Incorrectly Detected Paths• With our path crediting technique:
– We can distinguish hot paths in a regions
– May incorrectly detect hot paths in program
• May be crediting cold paths enough for them to seem hot compared to rest of program
Partial Path Count Matches Inc Total
ABDLMOP 100 ABDLMOPRSUVX
ABDLMOPRSUWX
ABDLMOPRSUVX
ABDLMOPRSUWX
+25
+25
+25
+25
25
25
25
25
Region 3
Region 2
Region 1
A
CB
D
L
NM
O
E
GF
HQP
R
TS
U
WV
X
JI
K
Y
Partial Path Length
• Length of Partial Paths drops drastically from splitting on function on loop back edges
Partial Path Length
0
10
20
30
40
50
60
70
80
164.gzip 175.vpr 177.mesa
179.art 181.mcf 183.equake 186.crafty 188.ammp 197.parser 256.bzip2 300.twolf
Length (# Ops)
Initial Length
Length After Splitting
Function Correlation
• MANY partial paths cross function boundaries– Should use function correlation
Function Boundaries Spanned By Partial Paths
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
164.gzip 175.vpr 177.mesa
179.art 181.mcf 183.equake 186.crafty 188.ammp 197.parser 256.bzip2 300.twolf
4 Function Boundaries3 Function Boundaries2 Function Boundaries1 Function Boundary0 Function Boundaries
Multiple Runs
• May be possible to use multiple runs to provide more accurate path profile data
164.gzip
0
500
1000
1500
2000
2500
3000
3500
4000
4500
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Number of Aggregated Runs
# Unique Paths
50K
100K
500K
1M
5M
10M
Future Work
• Region Formation– Characterize quality of our regions
• Important because no correlation between regions
– Regions stretching across function boundaries
• Noise Elimination– Crucial to removing false positives due to path
crediting
• Effects of Optimization– Find effects of superblocks, inlining, etc. on partial
paths and accuracy of path profile
Conclusion
• We introduce rationale and initial data of PMU-based path profiling
• PMU-based profiling shows promise
• At Sampling Period = 5M cycles– ~85% accurate– ~1% overhead
Questions?
References[Bala00]V. Bala, E. Duesterwald and S. Banerjia. “Dynamo: A Trasparent
Dynamic Optimization System” PLDI 2000.[Ball92]T. Ball and J.R. Larus. “Optimally Profiling and Tracing Programs”
TOPLAS 1992.[Ball96]T. Ball and J.R. Larus. “Efficient Path Profiling” MICRO-29, 1996.[Bond05] M.D. Bond and K.S. McKinley. “Practical Path Profiling for
Dynamic Optimizers”, CGO 2005.[Bruen03]D. Bruening, R. Garnett and S. Amarasinghe. “An Infrastructure
for Adaptive Dynamic Optimization” CGO 2003.[Chen03]H. Chen, W.C. Hsu, J. Lu, P.C. Yew and D.Y. Chen. “Dynamic
Trace Selection Using Performance Monitoring Hardware Sampling” CGO 2003.
[Conte94]T.M. Conte, B.A. Patel and J.S. Cox. “Using Branch Handling Hardware to Support Profile-Driven Optimization” MICRO-27, 1994.
References (cont)[Intel04]Intel, “Intel Itanium 2 Processor Reference Manual: For Software
Development and Optimization” May 2004.[Joshi04]R. Joshi, M.D. Bond and C. Zilles. “Targeted Path Profiling:
Lower Overhead Path Profiling for Staged Dynamic Optimization Systems” CGO 2004.
[Kistler01]T. Kistler and M. Franz. “Continuous Program Optimization” IEEE Trans. On Computers v50 no6 June 2001.
[Lu04]J. Lu, H. Chen, P.C. Yew and W.C. Hsu. “Design and Implementation of a Lightweight Dynamic Optimization System” Journal of ILP 6, 2004
[Merten00]M.C. Merten, A.R. Trick, E.M. Nystrom, R.D. Barnes, and W.W. Hwu. “A Hardware Mechanism for Dynamic Extraction and Relayout of Program Hot Spots” ISCA 2000.
[oicc] http://gelato.uiuc.edu[pin] http://rogue.colorado.edu/Pin
Extra Slides
ADORE Trace Selection
• Goal: Gather hot traces with many cache misses to add pre-fetches
• However, hot traces may not be enough to detect full hot paths
• Compiler can perform further analysis – Correlate BTB based traces into longer paths
PMU
Itanium 2
Sample
last 4 takenbranches
Branch Trace D1 Misses I1 Misses Cycles
10ac,640,66c,10c6
350 128 81280
… … … …
Branch Trace Table
BTB Trace
Hot Path
Partial Path Characteristics
• Partial Path extensions increase length ~20%• However, splitting drastically decreases lengths
– ~30% on function boundaries, ~20% more on loop back edges
• Many paths span 1 or more function boundaries– Indicates a great amount of function correlation is being thrown away
Benchmark Initial Ext Func Loop
164.gzip
175.vpr
177.mesa
179.art
181.mcf
183.equake
186.crafty
188.ammp
197.parser
256.bzip2
300.twolf
28.9
41.8
44.6
29.5
32.0
65.8
36.0
31.3
28.7
38.8
37.8
34.8
50.5
53.8
34.7
38.8
75.1
45.2
39.5
35.2
45.8
46.5
22.8
30.6
35.3
32.1
33.7
66.8
31.7
36.4
14.7
33.4
32.5
20.4
19.6
33.0
22.9
25.5
54.5
31.1
28.5
12.7
22.7
25.4
Benchmark 0 1 2 3 4
164.gzip
175.vpr
177.mesa
179.art
181.mcf
183.equake
186.crafty
188.ammp
197.parser
256.bzip2
300.twolf
187
234
75
230
1106
231
1367
298
488
532
1293
149
171
50
29
237
31
2281
123
654
353
514
35
162
27
7
96
24
968
42
599
279
467
2
65
7
1
26
1
154
5
224
55
55
0
12
0
0
1
0
13
2
40
9
7Function Boundaries SpannedAverage Partial Path Lengths