Does Training Input Selection Matter for Feedback-Directed

Optimizations?

Paul Berubeberube@cs.ualberta.ca

University of Alberta

CDP05, October 17, 2005

September 28, 2005 Paul Berube 2

Outline

• Background and motivation

• Aestimo: an FDO evaluation tool

• Workload Selection

• Results

September 28, 2005 Paul Berube 3

What Is FDO?

compile train

Feedback-Directed Optimization:

compile evaluate

September 28, 2005 Paul Berube 4

What Is FDO?

compile train

Feedback-Directed Optimization:

compile evaluate

traininginput

profileevalinput

September 28, 2005 Paul Berube 5

Performance Evaluation Space

Programs

Eva

luat

ion

Inpu

ts

Static optimization

September 28, 2005 Paul Berube 6

Performance Evaluation Space

Programs

Eva

luat

ion

Inpu

ts

Trai

ning

Inpu

ts

FDO

September 28, 2005 Paul Berube 7

Performance Evaluation Space

Programs

Eva

luat

ion

Inpu

ts

Trai

ning

Inpu

ts

Only 1 Train input

SPEC

Usually 1 Ref input

September 28, 2005 Paul Berube 8

The Big Question

• Does the selection of training inputs matter for feedback-directed optimization?– Different transformation decisions?– Different performance?

September 28, 2005 Paul Berube 9

Aestimo

• An FDO evaluation tool

• Automates training and evaluating on a large number of inputs

• Isolates individual transformations– Fewer experiment variables– Results vary by transformation

• Measures:– Differences in transformation decisions– Performance differences

September 28, 2005 Paul Berube 10

An Overview of Aestimo

Compile

BinariesOptimization

Logs

Execute

Analyze

ProgramWorkload

September 28, 2005 Paul Berube 11

An Overview of Aestimo

Compile

BinariesOptimization

Logs

Execute

Analyze

ProgramWorkload

One Per Input

One Per Input

September 28, 2005 Paul Berube 12

An Overview of Aestimo

Compile

BinariesOptimization

Logs

Execute

Analyze

ProgramWorkload

Binary X Input5 times each

September 28, 2005 Paul Berube 13

An Overview of Aestimo

Compile

BinariesOptimization

Logs

Execute

Analyze

ProgramWorkload

WorkloadPerformanc

e

September 28, 2005 Paul Berube 14

An Overview of Aestimo

Compile

BinariesOptimization

Logs

Execute

Analyze

ProgramWorkload

WorkloadPerformanc

eTransformatio

nDifferences

September 28, 2005 Paul Berube 15

An Overview of Aestimo

Compile

BinariesOptimization

Logs

Execute

Analyze

ProgramWorkload

WorkloadPerformanc

eTransformatio

nDifferences

FDO vs.Static

September 28, 2005 Paul Berube 16

An Overview of Aestimo

Compile

BinariesOptimization

Logs

Execute

Analyze

ProgramWorkload

WorkloadPerformanc

eTransformatio

nDifferences

FDO vs.Static Resubstitutio

n

September 28, 2005 Paul Berube 17

Compilation Process

StaticBinary

OptimizationLog

Source

Static Compile

September 28, 2005 Paul Berube 18

Compilation Process

Source

Static Compile

StaticBinary

OptimizationLog

TrainingInput

InstrumentedBinary

Instr. Compile

Training RunProfile

FDO Compile FDOBinary

September 28, 2005 Paul Berube 19

Compilation Process

Source

Static Compile

StaticBinary

OptimizationLog

Instr. Compile

Training RunProfile

FDO Compile FDOBinary

TrainingInput

OptimizationLog

Static Compile FinalBinary

InstrumentedBinary

September 28, 2005 Paul Berube 20

Compilation Process

Source

Static Compile

StaticBinary

OptimizationLog

Instr. CompileInstrumented

Binary

Training RunProfile

FDO Compile FDOBinary

TrainingInput

OptimizationLog

Static Compile FinalBinary

September 28, 2005 Paul Berube 21

Workload Selection

• SPEC CINT2000 Benchmark inputs– 8 programs, 32 input

• 84 Additional Inputs– Contacted benchmark authors– Varied representative inputs– Existing collections– Synthetic input generator

September 28, 2005 Paul Berube 22

Results

• ORC compiler

• Inlining and if conversion

• Itanium and Itanium 2 processors

September 28, 2005 Paul Berube 23

Workload Performance: bzip2

InliningItanium

September 28, 2005 Paul Berube 24

Workload Performance: bzip2Training Input Selection Matters!

InliningItanium

September 28, 2005 Paul Berube 25

Summary of Contributions

• Training input selection does impact optimization decisions and performance

• Aestimo:– Automates training and evaluating on a large

number of inputs– Isolates individual transformations

• A large collection of representative inputs for SPEC CINT2000 programs

September 28, 2005 Paul Berube 26

Thank You

Questions?

September 28, 2005 Paul Berube 27

Performance: bzip2 trained on xml

-6

-4

-2

0

2

4

6

8

10

12

Evaluation Input

% F

aste

r th

an S

tati

c

InliningItanium

September 28, 2005 Paul Berube 28

Performance: bzip2.combined

0

2

4

6

8

10

12

14

16

18

Evaluation Input

% F

aste

r th

en S

tatic

InliningItanium