1 tcss 342 lecture notes course overview, review of math concepts, algorithm analysis and big-oh...
TRANSCRIPT
![Page 1: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/1.jpg)
1
TCSS 342 Lecture NotesCourse Overview,
Review of Math Concepts,Algorithm Analysis and Big-Oh Notation
Weiss book, Chapter 5
These lecture notes are copyright (C) Marty Stepp, 2005. They may not be rehosted, sold, or modified without expressed permission from the author. All rights reserved.
![Page 2: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/2.jpg)
2
Lecture outline Introduction and course objectives
Mathematical background review exponents and logarithms arithmetic and geometric series
Algorithm analysis and Big-Oh notation the RAM model Big-Oh Big-Omega, Big-Theta, Little-Oh, Little-Omega
Comparison of algorithms to solve the Maximum Contiguous Subsequence Sum problem
![Page 3: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/3.jpg)
3
Course objectives (broad) prepare you to be a good software engineer
(specific) learn basic data structures and algorithms data structures – how data is organized algorithms – unambiguous sequence of steps to compute
something algorithm analysis – determining how long an algorithm will take to
solve a problem Who cares? Aren't computers fast enough and getting faster?
"Data Structures + Algorithms = Programs"-- Niklaus Wirth, author of Pascal language
![Page 4: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/4.jpg)
4
Given an array of 1,000,000 integers, find the maximum integer in the array.
Now suppose we are asked to find the kth largest element. (The Selection Problem)
…0 1 2 999,999
An example
![Page 5: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/5.jpg)
5
candidate solution 1 sort the entire array (from small to large), using Java's Arrays.sort()
pick out the (1,000,000 – k)th element
candidate solution 2 place the first k elements into a sorted array of size k for each of the remaining 1,000,000 – k elements,
keep the k largest in the array pick out the smallest of the k survivors and toss it
Candidate solutions
![Page 6: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/6.jpg)
6
Is either solution good? What advantages and disadvantages does each
solution have?
Is there a better solution?
What makes a solution "better" than another? Is it entirely based on runtime?
How would you go about determining which solution is better? could code them, test them could somehow make predictions and analysis of
each solution, without coding
![Page 7: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/7.jpg)
7
Why algorithm analysis? as computers get faster and problem sizes get bigger,
analysis will become more important
The difference between good and bad algorithms will get bigger
being able to analyze algorithms will help us identify good ones without having to program them and test them first
![Page 8: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/8.jpg)
8
Why data structures? when programming, you are an engineer
engineers have a bag of tools and tricks – and the knowledge of which tool is the right one for a given problem
Examples: arrays, lists, stacks, queues, trees, hash tables, heaps, graphs
![Page 9: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/9.jpg)
9
Mathematical background review
Let's review some math material that'll be useful later when computing the runtime of various algorithms.
![Page 10: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/10.jpg)
10
Math background: exponents Exponents
XY , or "X to the Yth power";X multiplied by itself Y times
Some useful identities XA XB = XA+B
XA / XB = XA-B
(XA)B = XAB
XN+XN = 2XN
2N+2N = 2N+1
![Page 11: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/11.jpg)
11
Math background:Logarithms Logarithms
definition: XA = B if and only if logX B = A intuition: logX B means:
"the power X must be raised to, to get B"
In this course, a logarithm with no base implies base 2.log B means log2 B
Examples log2 16 = 4 (because 24 = 16) log10 1000 = 3 (because 103 = 1000)
![Page 12: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/12.jpg)
12
Identities for logs with addition, multiplication, powers: log (AB) = log A + log B log (A/B) = log A – log B log (AB) = B log A
Identity for converting bases of a logarithm:
example:log432 = (log2 32) / (log2 4) = 5 / 2
1,0,,log
loglog ACBA
A
BB
C
CA
Logarithm identities
![Page 13: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/13.jpg)
13
Logarithm problem solving When presented with an expression of the form:
logaX = Y
and trying to solve for X, raise both sides to the a power.
X = aY
When presented with an expression of the form: logaX = logbY
and trying to solve for X, find a common base between the logarithms using the identity on the last slide.
logaX = logaY / logab
![Page 14: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/14.jpg)
14
Logarithm practice problems Determine the value of x in the following equation.
log7 x + log713 = 3
Determine the value of x in the following equation. log8 4 - log8 x = log8 5 + log16 6
![Page 15: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/15.jpg)
15
Series
for some expression Expr (possibly containing i ), means the sum of all values of Expr with each value of i between j and k inclusive
Example:
= (2(0) + 1) + (2(1) + 1) + (2(2) + 1) + (2(3) + 1) + (2(4) + 1)= 1 + 3 + 5 + 7 + 9= 25
Math review: Arithmetic series
k
ji
Expr
4
0
12i
i
![Page 16: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/16.jpg)
16
Series identities sum from 1 through N inclusive
is there an intuition for this identity? sum of all numbers from 1 to N
1 + 2 + 3 + ... + (N-2) + (N-1) + N
how many terms are in this sum? Can we rearrange them?
2
)1(
1
NNi
N
i
![Page 17: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/17.jpg)
17
More series identities sum from a through N inclusive
(when the series doesn't start at 1)
is there an intuition for this identity? The sum of all numbers from 4 to 10 is just the sum of all
numbers from 1 to 10, minus the sum of 1 through 3.
1
11
a
i
N
i
N
ai
iii
![Page 18: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/18.jpg)
18
Series of constants sum of constants
(when the body of the series doesn't contain the counter variable such as i)
example:
11
abkkkb
ai
b
ai
351410515510
4
10
4
ii
![Page 19: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/19.jpg)
19
Splitting series
for any constant k, splitting a sum with addition
moving out a constant multiple
b
ai
b
ai
b
ai
kiki
b
ai
b
ai
ikki
![Page 20: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/20.jpg)
20
Series of powers sum of powers of 2
1 + 2 + 4 + 8 + 16 + 32 = 64 - 1 = 63 think about binary representation of numbers...
111111 (63)+ 1 (1)1000000 (64)
when the series doesn't start at 0:
122 1
0
NN
i
i
1
00
222a
i
iN
i
iN
ai
i
![Page 21: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/21.jpg)
21
Series practice problems Give a closed form expression for the following
summation. A closed form expression is one without the or "…".
Give a closed form expression for the following summation.
2
0
2N
i
i
1
10
5N
i
i
![Page 22: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/22.jpg)
22
Algorithm analysis and Big-oh notation
![Page 23: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/23.jpg)
23
Algorithm performance How to determine how much time an algorithm A uses to solve
problem X ? Depends on input; use input size "N" as parameter Determine function f(N) representing cost
empirical analysis: code it and use timer running on many inputs
algorithm analysis: Analyze steps of algorithm, estimating amount of work each step takes
![Page 24: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/24.jpg)
24
Typically use a simple model for basic operation costs
RAM (Random Access Machine) model RAM model has all the basic operations:
+, -, *, / , =, comparisons fixed sized integers (e.g., 32-bit) infinite memory All basic operations take exactly one time unit (one CPU
instruction) to execute
RAM model
![Page 25: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/25.jpg)
25
Critique of the model Strengths:
simple easier to prove things about the model than the real machine can estimate algorithm behavior on any hardware/software
Weaknesses: not all operations take the same amount of time in a real
machine does not account for page faults, disk accesses, limited
memory, floating point math, etc
![Page 26: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/26.jpg)
26
Relative rates of growth most algorithms' runtime can be expressed as a
function of the input size N
rate of growth: measure of how quickly the graph of a function rises
goal: distinguish between fast- and slow-growing functions
we only care about very large input sizes(for small sizes, most any algorithm is fast enough)
this helps us discover which algorithms will run more quickly or slowly, for large input sizes
![Page 27: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/27.jpg)
27
Growth rate example
Consider these graphs of functions.Perhaps each one represents an algorithm:n3 + 2n2
100n2 + 1000
• Which growsfaster?
![Page 28: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/28.jpg)
28
Growth rate example
• How about now?
![Page 29: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/29.jpg)
29
Defn: T(N) = O(f(N))if there exist positive constants c , n0 such that: T(N) c · f(N) for all N n0
idea: We are concerned with how the function grows when N is large. We are not picky about constant factors: coarse distinctions among functions
Lingo: "T(N) grows no faster than f(N)."
Review: Big-Oh notation
![Page 30: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/30.jpg)
30
n = O(2n) ?
2n = O(n) ?
n = O(n2) ?
n2 = O(n) ?
n = O(1) ?
100 = O(n) ?
10 log n = O(n) ?
214n + 34 = O(2n2 + 8n) ?
Big-Oh example problems
![Page 31: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/31.jpg)
31
pick tightest bound. If f(N) = 5N, then:f(N) = O(N5)f(N) = O(N3)f(N) = O(N log N)f(N) = O(N) preferred
ignore constant factors and low order termsT(N) = O(N), not T(N) = O(5N)
T(N) = O(N3), not T(N) = O(N3 + N2 + N log N)
Wrong: f(N) O(g(N)) Wrong: f(N) O(g(N))
remove non-base-2 logarithmsf(N) = O(N log6 N)
f(N) = O(N log N) preferred
Preferred big-Oh usage
![Page 32: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/32.jpg)
32
Big-Oh of selected functions
![Page 33: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/33.jpg)
33
Ten-fold processor speedup "Who cares about choosing the fastest algorithm. If it is
too slow, we'll just upgrade the computer..."
Example data when computer is sped up by a factor of 10:
![Page 34: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/34.jpg)
34
Defn: T(N) = (g(N)) if there are positive constants c and n0 such that T(N) c g(N) for all N n0
Lingo: "T(N) grows no slower than g(N)."
Defn: T(N) = (g(N)) if and only if T(N) = O(g(N)) and T(N) = (g(N)).
Big-Oh, Omega, and Theta establish a relative ordering among all functions of N
Defn: T(N) = o(g(N)) if and only if T(N) = O(h(N)) and T(N) (h(N)).
Big omega, theta
![Page 35: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/35.jpg)
35
notation intuition
O (Big-Oh)
(Big-Omega)
(Theta) =
o (little-Oh) <
Intuition about the notations
![Page 36: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/36.jpg)
36
Fact: If f(N) = O(g(N)), then g(N) = (f(N)).
Proof: Suppose f(N) = O(g(N)).Then there exist constants c and n0 such that f(N) c g(N) for all N n0
Then g(N) (1/c) f(N) for all N n0, and so g(N) = (f(N))
More about asymptotics
![Page 37: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/37.jpg)
37
T(N) = O(f(N)) f(N) is an upper bound on T(N) T(N) grows no faster than f(N)
T(N) = (g(N)) g(N) is a lower bound on T(N) T(N) grows at least as fast as g(N)
T(N) = o(h(N)) T(N) grows strictly slower than h(N)
More terminology
![Page 38: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/38.jpg)
38
Facts about big-Oh
If T1(N) = O(f(N)) and T2(N) = O(g(N)), then T1(N) + T2(N) = O(f(N) + g(N)) T1(N) * T2(N) = O(f(N) * g(N))
If T(N) is a polynomial of degree k, then:T(N) = (Nk)
example: 17n3 + 2n2 + 4n + 1 = (n3)
logk N = O(N), for any constant k
![Page 39: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/39.jpg)
39
Hierarchy of Big-Oh Functions, ranked in increasing order of growth:
1 log log n log n n n log n n2
n2 log n n3
... 2n
n! nn
![Page 40: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/40.jpg)
40
Algebraex. f(N) = N / log N
g(N) = log Nsame as asking which grows faster, N or log2 N
Evaluate:
)(
)(lim
Ng
NfN
limit is Big-Oh relation
0 f(N) = o(g(N))
c 0 f(N) = (g(N))
g(N) = o(f(N))
no limit no relation
Techniques
![Page 41: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/41.jpg)
41
L'Hôpital's rule:
If and , then
example: f(N) = N, g(N) = log NUse L'Hôpital's rulef'(N) = 1, g'(N) = 1/N g(N) = o(f(N))
)(lim NfN
)(lim NgN
)('
)('lim
)(
)(lim
Ng
Nf
Ng
NfNN
Techniques, cont'd
![Page 42: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/42.jpg)
42
for (int i = 0; i < n; i += c) // O(n) statement(s); Adding to the loop counter means that the loop runtime grows linearly
when compared to its maximum value n. Loop executes its body exactly n / c times.
for (int i = 0; i < n; i *= c) // O(log n) statement(s); Multiplying the loop counter means that the maximum value n must grow
exponentially to linearly increase the loop runtime; therefore, it is logarithmic.
Loop executes its body exactly logc n times.
for (int i = 0; i < n * n; i += c) // O(n2) statement(s); The loop maximum is n2, so the runtime is quadratic.
Loop executes its body exactly (n2 / c) times.
Program loop runtimes
![Page 43: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/43.jpg)
43
Nesting loops multiplies their runtimes.for (int i = 0; i < n; i += c) { // O(n2) for (int j = 0; j < n; i += c) { statement;} }
Loops in sequence add together their runtimes, which means the loop set with the larger runtime dominates.for (int i = 0; i < n; i += c) { // O(n) statement;}for (int i = 0; i < n; i += c) { // O(n log n) for (int j = 0; j < n; i *= c) { statement;} }
More loop runtimes
![Page 44: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/44.jpg)
44
Loop runtime problems Compute the exact value of the variable sum after the
following code fragment, as a closed-form expression in terms of input size n.int sum = 0;
for (int i = 1; i <= n; i *= 2) {
sum++;
}
for (int i = 1; i <= 1000; i++) {
sum++;
sum++;
}
![Page 45: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/45.jpg)
45
Loop runtime problems Compute the exact value of the variable sum after the
following code fragment, as a closed-form expression in terms of input size n.int sum = 0;
for (int i = 1; i <= n; i++) {
for (int j = 1; j <= i / 2; j += 2) {
sum++;
}
}
![Page 46: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/46.jpg)
46
"Maximum subsequence sum" problem
![Page 47: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/47.jpg)
47
Maximum subsequence sum
The maximum subsequence sum problem:
Given a sequence of integers A0, A1, ..., An - 1,
find the maximum value of
for any integers 0 (i, j) < n.
(This sum is zero if all numbers in the sequence are negative.)
For example, the maximum subseq. sum of this list is 33:[2, 9, -4, 1, -20, 3, 15, -10, 20, 5, -100, 10, 7]
j
ikkA
![Page 48: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/48.jpg)
48
Thoughts about the problem This is a maximization problem.
There are a set of possible answers (sums of subsets of the original array), and we are being asked to find the largest one.
When solving maximization problems, one possible technique is to generate all possible solutions, compare them, and choose the appropriate one.
In the context of this problem, that would mean generating all possible subsets of the original array, adding them up to determine each of their sums, and comparing these sums to choose the best one of them all.
brute force algorithm: A method for solving a problem that proceeds in a simple and obvious way, possibly making it require more steps than necessary.
![Page 49: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/49.jpg)
49
First algorithm (brute force)generate and test all possible subsequences
// implement togetherfunction maxSubsequence(array[]): max sum = 0
for each starting index i, for each ending index j, add up the sum from Ai to Aj
if this sum is bigger than max, max sum = this sum
return max sum
Using the RAM model, how many steps does this algorithm require?
What do you believe is the runtime (Big-Oh) of this algorithm? How could we empirically test this?
What is wasteful about the algorithm? How could it be improved?
![Page 50: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/50.jpg)
50
More observations The preceding algorithm is a brute force algorithm, but
it isn't even the best brute force algorithm.
We are redundantly re-computing a large number of values many times.
For example, we compute the sum of the subsequence between indexes 2 and 5,A[2] + A[3] + A[4] + A[5]
next we compute the sum of the subsequence between indexes 2 and 6.A[2] + A[3] + A[4] + A[5] + A[6]
We already had computed the sum of 2--5, but we compute it again as part of the 2--6 computation.
Assuming from the RAM model that each addition takes 1 unit of time, we are wasting 4 add operations!
![Page 51: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/51.jpg)
51
still try all possible combinations, but don't redundantly add the sums
key observation:
in other words, we don't need to throw away partial sums
can we use this information to remove one of the loops from our algorithm?
// implement together
function maxSubsequence2(array[]):
What is the runtime (Big-Oh) of this new algorithm? Can it still be improved further?
1j
ikkj
j
ikk AAA
Second algorithm (improved)
![Page 52: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/52.jpg)
52
Observations: Claim 1 To make our code even faster, we must avoid trying all
possible combinations; to do this, we must find a way to broadly eliminate many potential combinations from consideration.
Claim #1: A subsequence with a negative sum cannot be the start of the maximum-sum subsequence.
![Page 53: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/53.jpg)
53
Claim #1, more formally: If Ai, j is a subsequence such that ,
then there is no q such that Ai,q is the maximum-sum subsequence.
Proof: (do it together in class)
Can this help us produce a better algorithm?
0
j
ikkA
Claim 1, continued
![Page 54: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/54.jpg)
54
Claim #2: when examining subsequences left-to-right, for some starting index i, if Ai,j becomes the first subsequence starting with i ,
such that
Then no part of Ai,j can be part of the maximum-sum subsequence.
(Why is this a stronger claim than Claim #1?)
Proof: (do it together in class)
0
j
ikkA
Claim 2
![Page 55: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/55.jpg)
55
Claim 2, continued
These figures show the possible contents of Ai,j
![Page 56: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/56.jpg)
56
Can we eliminate another loop from our algorithm?
// implement together
function maxSubsequence3(array[]):
What is its runtime (Big-Oh)?
Is there an even better algorithm than this third algorithm? Can you make a strong argument about why or why not?
Third (best) algorithm
![Page 57: 1 TCSS 342 Lecture Notes Course Overview, Review of Math Concepts, Algorithm Analysis and Big-Oh Notation Weiss book, Chapter 5 These lecture notes are](https://reader036.vdocuments.mx/reader036/viewer/2022062314/56649e045503460f94af07c0/html5/thumbnails/57.jpg)
57
Express the running time as f(N), where N is the size of the input
worst case: your enemy gets to pick the input
average case: need to assume a probability distribution on the inputs
amortized: your enemy gets to pick the inputs/operations, but you only have to guarantee speed over a large number of operations
Types of runtime analysis