discrete structures & algorithms functions & asymptotic complexity
TRANSCRIPT
2
To understand how to perform computations efficiently, we need to know– How to count!– How to sum a sequence of steps.– Reason about the complexity/efficiency of an
algorithm.– Prove the correctness of our analysis.
• Logic• Methods of inference• Proof strategies• Induction• …
– (and eventually) How to create better algorithms.
3
Functions
• We can represent the number of steps needed to achieve a goal as a function of the size of the problem.
• Next: A quick refresher on functions– You should know (most of this) from other
math/engineering courses.
4
Functions
Definition: A function f : A B is a subset of AxB where a A, b B unique and <a,b>
f.(A function is a mapping of elements from one set to another.)
5
Functions
Definition: a function f : A B is a subset of AxB where a A, b B unique and <a,b> f.
A
B
A
B
A point!
6
Functions
A = {Michael, Tito, Janet, Cindy, Bobby}B = {Katherine Evans, Carol Brady, Mother
Teresa}
Let f: A B be defined as f(a) = mother(a).
Michael Tito Janet Cindy Bobby
Katherine
Carol
Mother Teresa
7
Image and preimage
For any set S A, image(S) = {b : a S, f(a) = b}
So, image({Michael, Tito}) = {Katherine Evans} image(A) = B - {Mother Teresa}
image(S) = f(S)
Michael Tito
Janet Cindy Bobby
Katherine Evans
Carol Brady
Mother Teresa
8
Image and preimage
For any S B, preimage(S) = {a: b S, f(a) = b}
So, preimage({Carol Brady}) = {Cindy, Bobby} preimage(B) = A
preimage(S) = f-
1(S)
Michael Tito Janet Cindy Bobby
Katherine Evans
Carol Brady
Mother Teresa
9
Image and preimage
What is image(preimage(S))?a) Sb) { }c) subset of Sd) superset of S e) who knows?
S
Michael Tito Janet Cindy Bobby
Katherine Evans
Carol Brady
Mother Teresa
10
Functions and their properties
• You should be familiar with these properties (past classes)
or
• Functions derive many of their properties from sets (they are defined on sets)
• As you already know much about sets -> should be able to derive these properties
11
Functions and their properties
f(A B) f(A) f(B)?
f(A B) = {y : x (A B), f(x) = y}
Choose an arbitrary b f(A B), and show that it must also be an element of f(A) f(B).
So, x (A B) such that f(x) = y.
If x A (it is), then f(x) = y f(A).
If x B (it is), then f(x) = y f(B).
y f(A), and y f(B), so b f(A) f(B).
12
Injections
A function f: A B is one-to-one (injective, an injection) if a,b,c, (f(a) = b f(c) = b) a = c
Not one-to-one
Every b B has at most 1 preimage.
Michael Tito Janet Cindy Bobby
Katherine Evans
Carol Brady
Mother Teresa
13
Surjections
A function f: A B is surjective if b B, a A f(a) = b
Not surjective
Every b B has at least 1 preimage.
Michael Tito Janet Cindy Bobby
Katherine Evans
Carol Brady
Mother Teresa
14
Bijections
A function f: A B is bijective if it is both one-to-one (injective) and surjective.
Isaak Bri
Lynette
Aidan Evan
Cinda Dee Deb Katrina Dawn
Every b B has exactly 1 preimage.
An important implication of this
characteristic:The preimage (f-1)
is a function!
Isaak Bri
Lynette
Aidan Evan
Cinda Dee Deb Katrina Dawn
15
Examples
Suppose f: R+ R+, f(x) = x2.
Is f one-to-one?
Is f surjective?
Is f bijective?
yesyes
yes
18
Composition of functions
Let g:XY, and f:YZ be functions. Then the composition of f and g is:
(f o g)(x) = f(g(x))
g f
20
Asymptotic complexity
• Goal: Estimate properties (e.g., running time) of an algorithm as a function of input size n for large n.
• Expressed using only the highest-order term in the expression for the exact running time.– Instead of exact running time, say (n2).
• Describes behavior of function in the limit.
21
Asymptotic notation
• , O, , o, • Defined for functions over the natural numbers.
– Example: f(n) (n2).– Describes how f(n) grows in comparison to n2.
(n2) define a set of functions; in practice used to compare two function sizes.
• The notations describes the rate-of-growth for a set of functions.
22
-notation (big-Theta)
(g(n)) = {f(n) : positive constants c1, c2, n0, such that n n0,
we have c1g(n) f(n) c2g(n) }
For function g(n), we define (g(n)), big-Theta of n, as the set:
g(n) is an asymptotically tight bound for f(n).
Intuitively: Set of all functions that have the same rate of growth as g(n).
23
-notationFor function g(n), we define (g(n)), big-Theta of n, as the set:
g(n) is an asymptotically tight bound for f(n).
Technically, f(n) (g(n)).Older usage, f(n) = (g(n)).I’ll accept either…
(g(n)) = {f(n) : positive constants c1, c2, n0, such that n n0,
we have c1g(n) f(n) c2g(n) }
24
-notationFor function g(n), we define (g(n)), big-Theta of n, as the set:
If f(n) (g(n))
Then ? (0, ∞, or a constant )
(g(n)) = {f(n) : positive constants c1, c2, n0, such that n n0,
we have c1g(n) f(n) c2g(n) }
)(
)(lim
ng
nfn
25
Example
For f(n) = 10n2 - 3n (n2)• What constants for n0, c1, and c2 will work?• Make c1 a little smaller than the leading
coefficient, and c2 a little bigger.• To compare orders of growth, look at the
leading term.
Exercise: Prove that n2/2-3n (n2)
(g(n)) = {f(n) : positive constants c1, c2, and n0,
such that n n0, c1g(n) f(n) c2g(n)}
26
Example
• Does n5 (n4)?• Does 300n3 (n4)?• How about 22n (2n)?
• Does lg(n!) (n lg n)?– Proof using Stirling’s approximation (in the text)
for lg(n!).
(g(n)) = {f(n) : positive constants c1, c2, and n0, such that n n0, c1g(n) f(n)
c2g(n)}
27
O-notation
O(g(n)) = {f(n):
positive constants c and n0, such that n n0,
we have 0 f(n) cg(n) }
For function g(n), we define O(g(n)), big-O of n, as the set:
O(g(n)) is an asymptotic upper bound for f(n).
Intuitively: Set of all functions f whose rate of growth is the same as or lower than that of g(n).
f(n) = (g(n)) f(n) = O(g(n)).
(g(n)) O(g(n)).
?)(
)(lim
ng
nfn
28
Examples
• Any linear function an + b is in O(n2). How?
• Show that 3n3 O(n4) for appropriate c and n0.
O(g(n)) = {f(n) : positive constants c and n0, such that n n0, we have 0 f(n) cg(n) }
29
-notation (omega)
(g(n)) is an asymptotic lower bound for f(n).
Intuitively: Set of all functions whose rate of growth is the same as or higher than that of g(n).
f(n) = (g(n)) f(n) = (g(n)).
(g(n)) (g(n)).
(g(n)) = {f(n) : positive constants c and n0, such that n n0,
we have 0 cg(n) f(n)}
For function g(n), we define (g(n)), big-Omega of n, as the set:
?)(
)(lim
ng
nfn
30
Example
n (lg n). Choose c and n0.
(g(n)) = {f(n) : positive constants c and n0, such that n n0, we have 0 cg(n) f(n)}
32
Relations between , , O
(g(n)) = O(g(n)) (g(n))
• In practice, asymptotically tight bounds are obtained from asymptotic upper and lower bounds.
Theorem: For any two functions g(n) and f(n),
f(n) (g(n)) iff
f(n) O(g(n)) and f(n) (g(n)).
33
Example• Insertion sort takes (n2) in the worst case, so sorting
(as a problem) is O(n2). Why?
• Any sort algorithm must look at each item, so sorting is (n).
• In fact, using (e.g.) merge sort, sorting is (n lg n) in the worst case.
– No comparison sort to do better in the worst case. [We may not see a proof of this result in this course.]
34
Asymptotic notation in equations
• Can use asymptotic notation in equations to replace expressions containing lower-order terms.
• For example,4n3 + 3n2 + 2n + 1 = 4n3 + 3n2 + (n) = 4n3 + (n2) = (n3). How do we interpret
this?
• In equations, (f(n)) always stands for an anonymous function g(n) (f(n))– In the example above, (n2) stands for
3n2 + 2n + 1.
35
o-notation
f(n) becomes insignificant relative to g(n) as n approaches infinity:
lim [f(n) / g(n)] = 0.
n
g(n) is an upper bound for f(n) that is not asymptotically tight.
Observe the difference in this definition from previous ones. Why?
o(g(n)) = {f(n): c > 0, n0 > 0 such that n n0, we have 0 f(n) < cg(n)}.
For a given function g(n), the set little-o:
36
(g(n)) = {f(n): c > 0, n0 > 0 such that n n0, we have 0 cg(n) < f(n)}.
–notation (little-omega)
f(n) becomes arbitrarily large relative to g(n) as n approaches infinity:
lim [f(n) / g(n)] = .
n
g(n) is a lower bound for f(n) that is not asymptotically tight.
For a given function g(n), the set little-omega:
37
Limits• lim [f(n) / g(n)] = 0 f(n) (g(n)) n
• lim [f(n) / g(n)] < f(n) (g(n)) n
• 0 < lim [f(n) / g(n)] < f(n) (g(n)) n
• 0 < lim [f(n) / g(n)] f(n) (g(n)) n
• lim [f(n) / g(n)] = f(n) (g(n)) n
• lim [f(n) / g(n)] undefined can not say n
38
Relationships
• Subset relations between order-of-growth sets.
RR( f )O( f )
( f ) ( f )o( f )• f
39
Properties
• Transitivity– f(n) = (g(n)) & g(n) = (h(n)) f(n) = (h(n))– f(n) = O(g(n)) & g(n) = O(h(n)) f(n) = O(h(n))– f(n) = (g(n)) & g(n) = (h(n)) f(n) = (h(n))– f(n) = o (g(n)) & g(n) = o (h(n)) f(n) = o (h(n))– f(n) = (g(n)) & g(n) = (h(n)) f(n) = (h(n))
• Reflexivity– f(n) = (f(n))– f(n) = O(f(n))– f(n) = (f(n))
40
Properties• Symmetry
f(n) = (g(n)) iff g(n) = (f(n))
• Complementarity f(n) = O(g(n)) iff g(n) = (f(n)) f(n) = o(g(n)) iff g(n) = ((f(n))
43
Exponentials
• Useful (and elementary) Identities:
• Exponentials and polynomials
)(
0lim
nb
n
b
n
aon
an
nmnm
mnnm
aaa
aa
aa
)(
11
44
Logarithms
x = logba is the exponent for a = bx.
Natural log: ln a = logea
Binary log: lg a = log2a
lg2a = (lg a)2
lg lg a = lg (lg a)
ac
ab
bb
c
cb
bn
b
ccc
a
bb
b
ca
ba
aa
b
aa
ana
baab
ba
loglog
log
log
1log
log)/1(log
log
loglog
loglog
loglog)(log
45
Logarithms and exponentials – Bases
• If the base of a logarithm is changed from one constant to another, the value is altered by a constant factor.– Example: log2 n = log10 n * log210
– Base of logarithm is not an issue in asymptotic notation.
• Exponentials with different bases differ by a exponential factor (not a constant factor).– Example: 2n = (2/3)n *3n.
46
ExerciseExpress functions in A in asymptotic notation using functions in B.
A B
5n2 + 100n 3n2 + 2
A (n2), n2 (B) A (B)
log3(n2) log2(n3)
logba = logca / logcb; A = 2lgn / lg3, B = 3lgn, A/B =2/(3lg3)
nlg4 3lg n
alog b = blog a; B =3lg n=nlg 3; A/B =nlg(4/3) as n
lg2n n1/2
lim ( lga n / nb ) = 0 (here a = 2 and b = 1/2) A (B) n
A (B)
A (B)
A (B)
A (B)
47
Wrap-up
• What are the different asymptotic bounds on functions?
• How are the asymptotic bounds related?• Asymptotic bounds and algorithmic efficiency