performance measurement n assignment? n timing #include double when() { struct timeval tp;...

29
Performance Performance Measurement Measurement Assignment? Assignment? Timing Timing clude <sys/time.h> ble When() struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec * 1

Upload: pedro-penniston

Post on 01-Apr-2015

221 views

Category:

Documents


3 download

TRANSCRIPT

Page 1: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

Performance MeasurementPerformance Measurement

Assignment?Assignment? TimingTiming

#include <sys/time.h>double When(){

struct timeval tp;gettimeofday(&tp, NULL);return((double)tp.tv_sec + (double)tp.tv_usec * 1e-6);

}

Page 2: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

Paper SchedulePaper Schedule– 22 Students22 Students– 6 Days6 Days– Look at the schedule and email me your Look at the schedule and email me your

preference. Quickly.preference. Quickly.

Page 3: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

A Quantitative Basis for DesignA Quantitative Basis for Design

Parallel programming is an optimization Parallel programming is an optimization problem.problem.

Must take into account several factors:Must take into account several factors:– execution timeexecution time– scalabilityscalability– efficiencyefficiency

Page 4: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

A Quantitative Basis for DesignA Quantitative Basis for Design

Parallel programming is an optimization Parallel programming is an optimization problem.problem.

Must take into account several factors:Must take into account several factors: Also must take into account the costs:Also must take into account the costs:

– memory requirementsmemory requirements– implementation costsimplementation costs– maintenance costs etc.maintenance costs etc.

Page 5: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

A Quantitative Basis for DesignA Quantitative Basis for Design

Parallel programming is an optimization Parallel programming is an optimization problem.problem.

Must take into account several factors:Must take into account several factors: Also must take into account the costs:Also must take into account the costs: Mathematical performance models are used Mathematical performance models are used

to asses these costs and predict to asses these costs and predict performance.performance.

Page 6: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

Defining PerformanceDefining Performance

How do you define parallel performance?How do you define parallel performance? What do you define it in terms of?What do you define it in terms of? ConsiderConsider

– Distributed databasesDistributed databases– Image processing pipelineImage processing pipeline– Nuclear weapons testbedNuclear weapons testbed

Page 7: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

Amdahl's LawAmdahl's Law

Every algorithm has a sequential Every algorithm has a sequential component.component.

Sequential component limits speedupSequential component limits speedup

SequentialComponent

MaximumSpeedup

= 1/s = s

Page 8: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

Amdahl's LawAmdahl's Law

s

Speedup

Page 9: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

What's wrong?What's wrong?

Works fine for a given algorithm.Works fine for a given algorithm.– But what if we change the algorithm?But what if we change the algorithm?

We may change algorithms to increase We may change algorithms to increase parallelism and thus eventually increase parallelism and thus eventually increase performance.performance.– May introduce inefficiencyMay introduce inefficiency

Page 10: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

Metrics for PerformanceMetrics for Performance

SpeedupSpeedup EfficiencyEfficiency ScalabilityScalability Others …………..Others …………..

Page 11: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

SpeedupSpeedup

SpeedP

SpeedS

1

What is Speed?

What algorithm for Speed1?

What is the work performed?How much work?

Page 12: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

Two kinds of SpeedupTwo kinds of Speedup

RelativeRelative– Uses parallel algorithm on 1 processorUses parallel algorithm on 1 processor– Most commonMost common

AbsoluteAbsolute– Uses best known serial algorithmUses best known serial algorithm– Eliminates overheads in calculation.Eliminates overheads in calculation.

Page 13: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

SpeedupSpeedup

Algorithm AAlgorithm A– Serial execution time is 10 sec.Serial execution time is 10 sec.– Parallel execution time is 2 sec.Parallel execution time is 2 sec.

Algorithm BAlgorithm B– Serial execution time is 2 sec.Serial execution time is 2 sec.– Parallel execution time is 1 sec.Parallel execution time is 1 sec.

What if I told you A = B?What if I told you A = B?

Page 14: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec
Page 15: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

EfficiencyEfficiency

pS

E

The fraction of time a processor spends doing useful work

Page 16: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

Cost (Processor-Time Product)Cost (Processor-Time Product)

ppTC p = # processors

C

TE s

Page 17: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

Performance MeasurementPerformance Measurement

Algorithm X achieved speedup of 10.8 on Algorithm X achieved speedup of 10.8 on 12 processors.12 processors.– What is wrong?What is wrong?

A single point of reference is not enough!A single point of reference is not enough! What about asymptotic analysis?What about asymptotic analysis?

Page 18: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

Performance MeasurementPerformance Measurement

There is not a perfect way to measure and There is not a perfect way to measure and report performance.report performance.

Wall clock time seems to be the best.Wall clock time seems to be the best. But how much work do you do?But how much work do you do? Best Bet:Best Bet:

– Develop a model that fits experimental results.Develop a model that fits experimental results.

Page 19: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

Parallel Programming StepsParallel Programming Steps

Develop algorithmDevelop algorithm Develop a model to predict performanceDevelop a model to predict performance If the performance looks ok then codeIf the performance looks ok then code Check actual performance against modelCheck actual performance against model Report the performanceReport the performance

Page 20: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

Performance EvaluationPerformance Evaluation

Identify the dataIdentify the data Design the experiments to obtain the dataDesign the experiments to obtain the data Report dataReport data

Page 21: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

Performance EvaluationPerformance Evaluation

Identify the dataIdentify the data– Execution timeExecution time– Be sure to examine a range of data pointsBe sure to examine a range of data points

Design the experiments to obtain the dataDesign the experiments to obtain the data Report dataReport data

Page 22: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

Performance EvaluationPerformance Evaluation

Identify the dataIdentify the data Design the experiments to obtain the dataDesign the experiments to obtain the data

– Make sure the experiment measures what you Make sure the experiment measures what you intend to measure.intend to measure.

– Remember: Execution time is max time taken.Remember: Execution time is max time taken.– Repeat your experiments many timesRepeat your experiments many times– Validate data by designing a modelValidate data by designing a model

Report dataReport data

Page 23: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

Performance EvaluationPerformance Evaluation

Identify the dataIdentify the data Design the experiments to obtain the dataDesign the experiments to obtain the data Report dataReport data

– Report all information that affects executionReport all information that affects execution– Results should be separate from ConclusionsResults should be separate from Conclusions– Present the data in an easily understandable Present the data in an easily understandable

format.format.

Page 24: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

Finite Difference ExampleFinite Difference Example

Finite Difference CodeFinite Difference Code 512 x 512 x 5 Elements512 x 512 x 5 Elements 16 IBM RS6000 workstations16 IBM RS6000 workstations Connected via EthernetConnected via Ethernet

Page 25: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

Finite Difference ModelFinite Difference Model

Execution TimeExecution Time– ExTime = (Tcomp + Tcomm)/PExTime = (Tcomp + Tcomm)/P

Communication TimeCommunication Time– Tcomm = 2*lat + 4*bw*n*zTcomm = 2*lat + 4*bw*n*z

Computation TimeComputation Time– Estimate using some sample runsEstimate using some sample runs

Page 26: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

Estimated PerformanceEstimated Performance

Page 27: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

Finite Difference ExampleFinite Difference Example

Page 28: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

What was wrong?What was wrong?

EthernetEthernet Change the computation of TcommChange the computation of Tcomm

– Reduce the bandwithReduce the bandwith– Tcomm = 2*lat + 4*bw*n*z*P/2Tcomm = 2*lat + 4*bw*n*z*P/2

Page 29: Performance Measurement n Assignment? n Timing #include double When() { struct timeval tp; gettimeofday(&tp, NULL); return((double)tp.tv_sec + (double)tp.tv_usec

Finite Difference ExampleFinite Difference Example