algorithms recurrences. definition – a recurrence is an equation or inequality that describes a...
TRANSCRIPT
Algorithms
Recurrences
Recurrences
• Definition – a recurrence is an equation or inequality that describes a function in terms of its value on smaller inputs
• Example – recurrence for Merge-Sort
T(n)
(1) if n = 1
2T(n/2) + (n) if n > 1
Why Recurrences?• The complexity of many interesting
algorithms is easily expressed as a recurrence – especially divide and conquer algorithms
• The form of the algorithm often yields the form of the recurrence
• The complexity of recursive algorithms is readily expressed as a recurrence.
Why solve recurrences?
• To make it easier to compare the complexity of two algorithms
• To make it easier to compare the complexity of the algorithm to standard reference functions.
Example Recurrences for Algorithms
• Insertion sort
• Linear search of a list
otherwisen + 1)-T(n
1 n for 1)(nT
otherwise1 + 1)-T(n
1 n for 1)(nT
Recurrences for Algorithms,continued
• Binary search
T(n)
1 for n 1
T(n/2) + 1 otherwise
Casual About Some Details• Boundary conditions
– These are usually constant for small n– We sometimes use these to fill in the details of a
“rough guess” solution• Floors and ceilings
– Usually makes no difference in solution– Usually assume n is an “appropriate” integer (i.e. a
power of 2) and assume that the function behaves the same way if floors and ceilings were taken into consideration
Merge Sort Assumptions
• Actual recurrence is:
• But we typically assume that n = 2k where k is an integer and use the simpler recurrence.
T(n)
1 for n 1
T( n/2 ) + T( n/2 + n otherwise
Methods for Solving Recurrences
• Substitution (constructive induction)• Iterating the recurrence• Master Theorem
Substitution Method(Constructive Induction)
• Use mathematical induction to derive an answer
• Steps1. Guess the form of the solution2. Use mathematical induction to find constants or
show that they can be found and to prove that the answer is correct
Substitution
• GoalDerive a function of n (or other variables used to express the size of the problem) that is not a recurrence so we can establish an upper and/or lower bound on the recurrence
May get an exact solution or may just get upper or lower bounds on the solution
Constructive Induction• Suppose T includes a parameter n and n is a
natural number (positive integer)• Instead of proving directly that T holds for
all values of n, prove– T holds for a base case b (often n = 1)– For every n > b, if T holds for n-1, then T holds
for n. • Assume T holds for n-1• Prove that T holds for n follows from this
assumption
Example 1• Given
• Prove T(n) O(n2)– Note that this is the recurrence for insertion
sort and we have already shown that this is O(n2) using other methods
T(n)
1 for n 1
T(n - 1) + n otherwise
T(n) i
n(n 1)
2i1
n O(n2 )
Proof for Example 1• Guess that the solution for T(n) is a
quadratic equation • Base case T(1) = a + b + c. We need to find
values for a, b, and c first.• Assume this solution holds for n-1
• Now consider the case for n. Begin with the recurrence for T(n)
T(n)an2 bn c
T(n 1)a(n 1)2 b(n 1) c
T(n)T (n 1) n
2/1or 0get we from
2/1or ,012get we21 from
and calculatecan we theseFrom
21
that case thebemust
it then =T(n) conclude toare weIf
)()21(
2=
)1()1()(
assumptionBy
)1()(
2
2
2
2
bbacbac
aabab
ba
cbacbabaa
cbnan
cbanbaan
ncbbnaanan
ncnbnanT
nnTnT
1for )()()(
and 0=c ,1for 1)( sincebut
c 1 =
)1()1()1(
1for 1)( and )()()(
c.for valueaget torecurrence theof definition
theand hypothesis new thisusecan weand hypothesis
complete more a have now But we not. is for value
but the d,constraine now are and for valuesThe
212
21
212
21
212
21
nnnnT
nnT
cT
nnTcnnnT
c
ba
Example 2– Establishing an Upper Bound
3
3
3
0
3
0
3
3
)()( :need what wequitenot is This
2/1
)2/(4
)2/(4)(
)(for recurrence with theStarting
)2/()2/( Assume
)( that prove want to wecase In this
integeran is where2 :Assumption
)()( :Guess
)2/(4)( :Recurrence
ncnT
ncn
nnc
nnTnT
nT
nnncnT
nncnnT
kn
nOnT
nnTnT
k
>0an greater th something< - >you wantanswer <
form in the expression the writetry to–heuristic General
1 and 2
)(
)(
)(
Trick
2/1 )(
)( that prove want toWe
3
32
13
32
13
32
1
3
0
3
nccn
ncncn
ncncn
ncnnT
ncnnT
nncnnT
example.next theSee bound. tight anot is
itbut bound,upper an destablishe have that weNote
1 ofcondition boundary afor 3
aselect can wecase In this condition.boundary
specified asatisfy enough to large is that value aSelect
1 and 2 )( shown that have We
specifiedcondition boundary a need still We3
nc
c
nccnnT
Ex. 3–Fallacious Argument
base) n allfor hold(must 0for holdsonly
because incorrect, is But this )()(
)2/(4
)2/(4)(
)(for recurrence with theStarting
)2/()2/( Assume
)( that prove want to wecase In this
)integer an for 2 (Assume )()( :Guess
)2/(4)( :Recurrence
222
2
2
0
2
0
2
2
n
cnncnnOnT
ncn
nnc
nnTnT
nT
nnncnT
nncnnT
knnOnT
nnTnTk
Ex. 3–Try again
)2/()2/()2/( Assume
)( Guess
order termlower asubstract you
in which hypothesis inductive new aover with start
>+ something< + >you want term< )(
situation in the yourself findyou When
Heuristic
hypothesis inductive theRevise
)(
point thisget toyou When
2
2
1
2
2
1
2
ncncnT
ncncnT
nT
ncnnT
Ex. 3–Try again cont.
.conditions
initial thehandle enough to large be toSelect
1 allfor )(
so 1 of valuesallfor positive is last term
theand formcorrect in the are termsfirst two theNow
)(
2
))2/()2/(4(
)2/(4)(
recurrence with Starting
1
22
2
1
2
22
2
1
2
2
1
2
2
1
c
cncncnT
c
nncncnc
nncnc
nncnc
nnTnT
Boundary Conditions• Boundary conditions are not usually
important because we don’t need an actual c value (if polynomially bounded)
• But sometimes it makes a big difference– Exponential solutions– Suppose we are searching for a solution to:
and we find the partial solution T(n)T (n / 2)2
T(n)cn
Boundary Conditions cont.
)1()1()(1)1(
1 = T(1) if dramatic moreeven are results The
).2()3( And
).3()( that implies this
3)1(
iscondition boundary theifBut
).2()( that implies this
2)1(
iscondition boundary theIf
n
nn
n
n
nTT
nT
T
nT
T
Boundary Conditions
• The solutions to the recurrences below have very different upper bounds
T(n)1 for n = 1
T (n/2)2 otherwise
T(n)2 for n = 1
T (n/2)2 otherwise
T(n)3 for n = 1
T (n/2)2 otherwise
Changing Variables
mmSmS
T
m)T()T(
nnm
n
m
mm
m
)2/(2)(
recurrence new get the to)2( S(m) Rename
222
2or lg Rename
integers. to
roundingabout t worry won':Assumption
lgn2TT(n)
one.familiar more a to
recurrence a change sometimesCan
2
Changing Variables continued
)lglglg
)lg()()2()(
obtain weT(n), toS(m) from Changing
)lg()(
solvedalready have werecurrence a is
)2/(2)(
nnO(
mmOmSTnT
mmOmS
mmSmS
m
Iterating the Recurrence
• The math can be messy with this method• Can sometimes use this method to get an
estimate that we can use for the substitution method
Example 4
nith
nTnnn
nTnnnnT
nTnn
nTnnnT
nTnnT
i2 is series in the term that theobserve We
)8/(6442
))8/(44/(162)(
again recurrence theIterate
)4/(162
))4/(42/(4)(
recurrence theiteratingStart
)2/(4)(
21lg
2
2lg
loglog
lg
i
2...42
...42
...42)(
hat Remember t
)1(2...42)(
as series therewritecan we
2 is term that theknow wesince Now,
lg then n,2or ,12/When
1.2/reach when webe will
it condition,boundary a as 1 use weIf condition?
boundary areach webefore iterate wedofar How
nnnnn
nnnn
nnnnnnT
na
nTnnnnT
nith
nin
n
n
an
n
ii
i
bb
)(
)(+ )(=
)()1(
)(+ 1212
=
T(1))2+...2+2+2()(
1for x 11
x
nprogressio geometric aout Factor
)1(2...42)(
2
22
2
2
lg
21-lg210
n
0=i
1
k
21lg
n
nn
nnn
nn
nnnTx
x
TnnnnnnT
n
n
n
n
Recurrence Trees• Allow you to visualize the process of
iterating the recurrence• Allows you make a good guess for the
substitution method• Or to organize the bookkeeping for
iterating the recurrence• Example
2)2/(2)( nnTnT
n2
T(n/2) T(n/2)
n2
(n/2)2 (n/2)2
T(n/4) T(n/4) T(n/4) T(n/4)
n2
(n/2)2 (n/2)2
(n/4)2 (n/4)2 (n/4)2 (n/4)2
Counting things
.recurrenceour in series
on the boundupper an is thisSo
2 ....1/41/21
is series the1/2 When x 1
1
1xfor
44) (page series harmonic Inverse
21
1
0
xx
k
k
Recurrence Tree Examplen2
(n/2)2 (n/2)2
(n/4)2 (n/4)2 (n/4)2 (n/4)2
lg n
Total: (n2)