cs2010: algorithms and data structures - lecture 2: analysis of … · 2018. 9. 14. · n 2.999...
TRANSCRIPT
![Page 1: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/1.jpg)
cs2010: algorithms and data structuresLecture 2: Analysis of Algorithms
Vasileios Koutavas
School of Computer Science and StatisticsTrinity College Dublin
![Page 2: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/2.jpg)
last lecture
Programs = Algorithms + Data structures.
→ Must be correct
→ Must be efficient
→ What is efficiency?→ Analysis of Algorithms: How to evaluate efficiency?
→ Established measure of efficiency (from now on performance):how the program scales to larger inputs. Measure the effect ofdoubling the input size to→ the running time of the algorithm→ the memory footprint of the algorithm
→ We want to be able to analyse algorithms w.r.t. their performance.
1
![Page 3: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/3.jpg)
last lecture
Programs = Algorithms + Data structures.
→ Must be correct
→ Must be efficient→ What is efficiency?→ Analysis of Algorithms: How to evaluate efficiency?
→ Established measure of efficiency (from now on performance):how the program scales to larger inputs. Measure the effect ofdoubling the input size to→ the running time of the algorithm→ the memory footprint of the algorithm
→ We want to be able to analyse algorithms w.r.t. their performance.
1
![Page 4: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/4.jpg)
last lecture
Programs = Algorithms + Data structures.
→ Must be correct
→ Must be efficient→ What is efficiency?→ Analysis of Algorithms: How to evaluate efficiency?
→ Established measure of efficiency (from now on performance):how the program scales to larger inputs. Measure the effect ofdoubling the input size to→ the running time of the algorithm→ the memory footprint of the algorithm
→ We want to be able to analyse algorithms w.r.t. their performance.
1
![Page 5: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/5.jpg)
last lecture
Programs = Algorithms + Data structures.
→ Must be correct
→ Must be efficient→ What is efficiency?→ Analysis of Algorithms: How to evaluate efficiency?
→ Established measure of efficiency (from now on performance):how the program scales to larger inputs. Measure the effect ofdoubling the input size to→ the running time of the algorithm→ the memory footprint of the algorithm
→ We want to be able to analyse algorithms w.r.t. their performance.
1
![Page 6: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/6.jpg)
why analyse algorithms?
→ Good programmer: to predict the performance of our programs.
→ Good client: to choose between alternativealgorithms/implementations.
→ Good manager: to provide guarantees to clients / avoid clientcomplaints.
→ Good scientist: to understand the nature of computing.
2
![Page 7: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/7.jpg)
example: linear search
Linear Search: simple search in an array:
boolean linearSearch1(int[] ar, int s) boolean found = false;for (int i = 0; i < ar.length; i++) if (ar[i] == s) found = true;
return found;
How can we evaluate how well the running time of this algorithmscales to larger inputs?
→ Evaluate1 its running time for various sizes of array ar.Then calculate the rate of growth of the running time withrespect to the input size (stay tuned).
1We will see a number of approaches how to evaluate running times3
![Page 8: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/8.jpg)
example: linear search
Linear Search: simple search in an array:
boolean linearSearch1(int[] ar, int s) boolean found = false;for (int i = 0; i < ar.length; i++) if (ar[i] == s) found = true;
return found;
How can we evaluate how well the running time of this algorithmscales to larger inputs?
→ Evaluate1 its running time for various sizes of array ar.Then calculate the rate of growth of the running time withrespect to the input size (stay tuned).
1We will see a number of approaches how to evaluate running times3
![Page 9: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/9.jpg)
example: linear search 2
Improved Linear Search: return as soon as possible.boolean linearSearch1(int[] ar, int s) for (int i = 0; i < ar.length; i++) if (ar[i] == s) return true;
return false;
Consider inputs arrays of size 100. Is the running time the same?
NO! Even for the same input size, some inputs will require more timethan others.
→ best case inputs; e.g: ar=[1,2,...,100], s = 1→ worst case inputs; e.g: ar=[1,2,...,100], s = 101→ everything in between (sometimes we talk about average case)
In this module (and usually in programming practice) we focus onthe worst-case inputs.
4
![Page 10: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/10.jpg)
example: linear search 2
Improved Linear Search: return as soon as possible.boolean linearSearch1(int[] ar, int s) for (int i = 0; i < ar.length; i++) if (ar[i] == s) return true;
return false;
Consider inputs arrays of size 100. Is the running time the same?
NO! Even for the same input size, some inputs will require more timethan others.
→ best case inputs; e.g: ar=[1,2,...,100], s = 1→ worst case inputs; e.g: ar=[1,2,...,100], s = 101→ everything in between (sometimes we talk about average case)
In this module (and usually in programming practice) we focus onthe worst-case inputs.
4
![Page 11: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/11.jpg)
example: linear search 2
Improved Linear Search: return as soon as possible.boolean linearSearch1(int[] ar, int s) for (int i = 0; i < ar.length; i++) if (ar[i] == s) return true;
return false;
Consider inputs arrays of size 100. Is the running time the same?
NO! Even for the same input size, some inputs will require more timethan others.
→ best case inputs; e.g: ar=[1,2,...,100], s = 1→ worst case inputs; e.g: ar=[1,2,...,100], s = 101→ everything in between (sometimes we talk about average case)
In this module (and usually in programming practice) we focus onthe worst-case inputs.
4
![Page 12: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/12.jpg)
evaluating running times
In the course of the next lectures we will see the following methodsfor evaluating how an algorithm’s running time scales:
1. Scientific method: measure running times through experiments
2. Mathematical methods: consider a convenient model ofcomputation and:→ sum up the number of program steps for worst-case inputs of size
N (N is a variable).→ calculate the number of program steps for worst-case inputs of
size N, when N nears infinity.This is the asymptotic (worst-case) running time — notation big-O,Ω, Θ.
5
![Page 13: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/13.jpg)
today
ROBERT SEDGEWICK | KEVIN WAYNE
F O U R T H E D I T I O N
Algorithms
http://algs4.cs.princeton.edu
Algorithms ROBERT SEDGEWICK | KEVIN WAYNE
‣ introduction
‣ observations
‣ mathematical models
‣ order-of-growth classifications
‣ theory of algorithms
‣ memory
1.4 ANALYSIS OF ALGORITHMS
→ Parts from S&W 1.4→ Estimate the performance of algorithms by
→ Experiments & Observations→ Precise Mathematical Calculations
6
![Page 14: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/14.jpg)
3
Running time
Analytic Engine
“ As soon as an Analytic Engine exists, it will necessarily guide the future course of the science. Whenever any result is sought by its aid, the question will arise—By what course of calculation can these results be arrived at by the machine in the shortest time? ” — Charles Babbage (1864)
how many times do you
have to turn the crank?
![Page 15: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/15.jpg)
6
Some algorithmic successes
Discrete Fourier transform.
・Break down waveform of N samples into periodic components.
・Applications: DVD, JPEG, MRI, astrophysics, ….
・Brute force: N 2 steps.
・FFT algorithm: N log N steps, enables new technology.
Friedrich Gauss1805
8T
16T
32T
64T
time
1K 2K 4K 8Ksize
quadratic
linearithmic
linear
![Page 16: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/16.jpg)
7
Some algorithmic successes
N-body simulation.
・Simulate gravitational interactions among N bodies.
・Brute force: N 2 steps.
・Barnes-Hut algorithm: N log N steps, enables new research. Andrew AppelPU '81
8T
16T
32T
64T
time
1K 2K 4K 8Ksize
quadratic
linearithmic
linear
![Page 17: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/17.jpg)
Q. Will my program be able to solve a large practical input?
Insight. [Knuth 1970s] Use scientific method to understand performance.
The challenge
8
Why is my program so slow ? Why does it run out of memory ?
![Page 18: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/18.jpg)
9
Scientific method applied to analysis of algorithms
A framework for predicting performance and comparing algorithms.
Scientific method.
・Observe some feature of the natural world.
・Hypothesize a model that is consistent with the observations.
・Predict events using the hypothesis.
・Verify the predictions by making further observations.
・Validate by repeating until the hypothesis and observations agree.
Principles.
・Experiments must be reproducible.
・Hypotheses must be falsifiable.
Feature of the natural world. Computer itself.
![Page 19: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/19.jpg)
Scientific Approach:
Evaluating Performance byExperiments
![Page 20: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/20.jpg)
11
Example: 3-SUM
3-SUM. Given N distinct integers, how many triples sum to exactly zero?
Context. Deeply related to problems in computational geometry.
% more 8ints.txt830 -40 -20 -10 40 0 10 5
% java ThreeSum 8ints.txt4
a[i] a[j] a[k] sum
30 -40 10 0
30 -20 -10 0
-40 40 0 0
-10 0 10 0
1
2
3
4
![Page 21: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/21.jpg)
public class ThreeSum public static int count(int[] a) int N = a.length; int count = 0; for (int i = 0; i < N; i++) for (int j = i+1; j < N; j++) for (int k = j+1; k < N; k++) if (a[i] + a[j] + a[k] == 0) count++; return count;
public static void main(String[] args) In in = new In(args[0]); int[] a = in.readAllInts(); StdOut.println(count(a));
12
3-SUM: brute-force algorithm
check each triple
for simplicity, ignore
integer overflow
![Page 22: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/22.jpg)
question
Suppose we care only about 100-element arrays.
Q. What is a worst-case input for ThreeSum?
A. They are all worst case inputs. For-loops always run to the end.
8
![Page 23: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/23.jpg)
question
Suppose we care only about 100-element arrays.
Q. What is a worst-case input for ThreeSum?
A. They are all worst case inputs. For-loops always run to the end.
8
![Page 24: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/24.jpg)
Q. How to time a program?
A. Manual.
13
Measuring the running time
% java ThreeSum 1Kints.txt
70
% java ThreeSum 2Kints.txt
% java ThreeSum 4Kints.txt
528
4039
tick tick tick
Observing the running time of a program
tick tick tick tick tick tick tick ticktick tick tick tick tick tick tick ticktick tick tick tick tick tick tick tick
tick tick tick tick tick tick tick ticktick tick tick tick tick tick tick ticktick tick tick tick tick tick tick ticktick tick tick tick tick tick tick ticktick tick tick tick tick tick tick ticktick tick tick tick tick tick tick ticktick tick tick tick tick tick tick ticktick tick tick tick tick tick tick ticktick tick tick tick tick tick tick ticktick tick tick tick tick tick tick ticktick tick tick tick tick tick tick ticktick tick tick tick tick tick tick ticktick tick tick tick tick tick tick ticktick tick tick tick tick tick tick ticktick tick tick tick tick tick tick ticktick tick tick tick tick tick tick ticktick tick tick tick tick tick tick ticktick tick tick tick tick tick tick ticktick tick tick tick tick tick tick ticktick tick tick tick tick tick tick ticktick tick tick tick tick tick tick ticktick tick tick tick tick tick tick ticktick tick tick tick tick tick tick ticktick tick tick tick tick tick tick tick
![Page 25: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/25.jpg)
Q. How to time a program?
A. Automatic.
14
Measuring the running time
client code
public static void main(String[] args) In in = new In(args[0]); int[] a = in.readAllInts(); Stopwatch stopwatch = new Stopwatch(); StdOut.println(ThreeSum.count(a)); double time = stopwatch.elapsedTime(); StdOut.println("elapsed time " + time);
public class Stopwatch public class Stopwatch
Stopwatch()Stopwatch() create a new stopwatch
double elapsedTime()elapsedTime() time since creation (in seconds)
(part of stdlib.jar )
![Page 26: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/26.jpg)
Run the program for various input sizes and measure running time.
15
Empirical analysis
![Page 27: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/27.jpg)
Run the program for various input sizes and measure running time.
16
Empirical analysis
N time (seconds) †
250 0.0
500 0.0
1,000 0.1
2,000 0.8
4,000 6.4
8,000 51.1
16,000 ?
![Page 28: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/28.jpg)
Standard plot. Plot running time T (N) vs. input size N.
17
Data analysis
1K
.1
.2
.4
.8
1.6
3.2
6.4
12.8
25.6
51.2
Analysis of experimental data (the running time of ThreeSum)
log-log plotstandard plot
lgNproblem size N2K 4K 8K
lg(T
(N))
runn
ing
tim
e T
(N)
1K
10
20
30
40
50
2K 4K 8K
straight lineof slope 3
![Page 29: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/29.jpg)
Log-log plot. Plot running time T (N) vs. input size N using log-log scale.
Regression. Fit straight line through data points: a N b.Hypothesis. The running time is about 1.006 × 10 –10 × N 2.999 seconds.
18
Data analysis
power law
slope
1K
.1
.2
.4
.8
1.6
3.2
6.4
12.8
25.6
51.2
Analysis of experimental data (the running time of ThreeSum)
log-log plotstandard plot
lgNproblem size N2K 4K 8K
lg(T
(N))
runn
ing
tim
e T
(N)
1K
10
20
30
40
50
2K 4K 8K
straight lineof slope 3
lg(T (N)) = b lg N + cb = 2.999c = -33.2103
T (N) = a N b, where a = 2 c
3 orders
of magnitude
![Page 30: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/30.jpg)
Try out the scientific analysis through experiments:
https://docs.google.com/spreadsheets/d/1WnihyK6g1pYdcT2ndZOqNNRkTitXkWKnOrTgCnM-bw8/edit?usp=sharing
9
![Page 31: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/31.jpg)
19
Prediction and validation
Hypothesis. The running time is about 1.006 × 10 –10 × N 2.999 seconds.
Predictions.
・51.0 seconds for N = 8,000.
・408.1 seconds for N = 16,000.
Observations.
validates hypothesis!
N time (seconds) †
8,000 51.1
8,000 51.0
8,000 51.1
16,000 410.8
"order of growth" of running
time is about N3 [stay tuned]
![Page 32: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/32.jpg)
Doubling hypothesis. Quick way to estimate b in a power-law relationship.
Run program, doubling the size of the input.
Hypothesis. Running time is about a N b with b = lg ratio. Caveat. Cannot identify logarithmic factors with doubling hypothesis.
20
Doubling hypothesis
N time (seconds) † ratio lg ratio
250 0.0 –
500 0.0 4.8 2.3
1,000 0.1 6.9 2.8
2,000 0.8 7.7 2.9
4,000 6.4 8.0 3.0
8,000 51.1 8.0 3.0
seems to converge to a constant b ≈ 3
lg (6.4 / 0.8) = 3.0
T (2N)
T (N)=
a(2N)b
aN b
= 2b
![Page 33: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/33.jpg)
21
Doubling hypothesis
Doubling hypothesis. Quick way to estimate b in a power-law relationship.
Q. How to estimate a (assuming we know b) ?
A. Run the program (for a sufficient large value of N) and solve for a.
Hypothesis. Running time is about 0.998 × 10 –10 × N 3 seconds.
N time (seconds) †
8,000 51.1
8,000 51.0
8,000 51.1
51.1 = a × 80003
⇒ a = 0.998 × 10 –10
almost identical hypothesis
to one obtained via linear regression
![Page 34: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/34.jpg)
22
Experimental algorithmics
System independent effects.
・Algorithm.
・Input data.
System dependent effects.
・Hardware: CPU, memory, cache, …
・Software: compiler, interpreter, garbage collector, …
・System: operating system, network, other apps, …
Bad news. Difficult to get precise measurements.
Good news. Much easier and cheaper than other sciences.
e.g., can run huge number of experiments
determines constant
in power law
determines exponent
in power law
![Page 35: CS2010: Algorithms and Data Structures - Lecture 2: Analysis of … · 2018. 9. 14. · N 2.999 seconds. 18 Data analysis power law slope 1K.1.2.4.8 1.6 3.2 6.4 12.8 25.6 51.2 Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081410/609f24a0b4838c661205a0dc/html5/thumbnails/35.jpg)
This was the scientific approach to algorithm analysis.
In the mathematical approach we do calculations instead ofexperiments.
10