4.4 the recursion-tree methodelomaa/teach/aads-2015-3.pdf9/10/2015 1 4.4 the recursion-tree method...
TRANSCRIPT
9/10/2015
1
4.4 The recursion-tree method
• Let us see how a recursion tree wouldprovide a good guess for the recurrence
= 3 4 )• Start by nding an upper bound• Floors and ceilings usually do not matter
when solving recurrences• Create a recursion tree for the recurrence
= 3 4 having written out theimplied constant coef cient > 0
10-Sep-15MAT-72006 AADS, Fall 2015 128
10-Sep-15MAT-72006 AADS, Fall 2015 129
9/10/2015
2
10-Sep-15MAT-72006 AADS, Fall 2015 130
• Subproblem sizes decrease by a factor of 4each time we go down one level =>– we eventually must reach a boundary
condition• Subproblem size for a node at depth is /4
– It hits = 1 when /4 = 1 or, equivalently,when = log
• Thus, the tree has log + 1 levels (at depths0, 1, 2, … , log )
10-Sep-15MAT-72006 AADS, Fall 2015 131
9/10/2015
3
• Each level has 3 × the nodes of level above– the number of nodes at depth is 3
• Subproblem sizes reduce by a factor of 4 foreach level we go down, each node at depth ,
= 0,1, 2, … , log 1, has cost of 4 )• Multiplying, we see that the total cost over all
nodes at depth is 3 4 ) = (3 16)• The bottom level, at depth log , has 3 =
nodes, each contributing cost (1), for atotal cost of (1), which is ), since
(1) is a constant
10-Sep-15MAT-72006 AADS, Fall 2015 132
• Add up the costs over all levels to determinethe cost for the entire tree:
=3
16 )
=(3 16) 1
(3 16) 1 )
By the value of a geometric (or exponential)series
=1
110-Sep-15MAT-72006 AADS, Fall 2015 133
9/10/2015
4
• We can use an in nite decreasing geometricseries as an upper bound
=3
16 )
316
=1
(3 16)
=1613 )
10-Sep-15MAT-72006 AADS, Fall 2015 134
• We have derived a guess of forour recurrence = 3 4 )
• Root of the tree contributes to the totalcost – a constant fraction of the total cost– The cost of the root dominates the total cost
• If is indeed an upper bound for therecurrence, then it must be a tight bound
• First recursive call contributes cost of ),must be a lower bound for the
recurrence10-Sep-15MAT-72006 AADS, Fall 2015 135
9/10/2015
5
• Use the substitution method to verify that theguess was correct, i.e., is anupper bound for the recurrence
• We want to show that for someconstant > 0
= 3 444)
=3
16
as long as (16 13)10-Sep-15MAT-72006 AADS, Fall 2015 136
• A more intricate example is the recurrence) = /3) + (2 /3) + )
• represents the constant in the term• Adding the values across the levels of the
recursion tree yields value for every level• The longest simple path from the root to a
leaf is (2 3 (2 3) 1• (2 3) = 1 when = log• The height of the tree is log
10-Sep-15MAT-72006 AADS, Fall 2015 137
9/10/2015
6
10-Sep-15MAT-72006 AADS, Fall 2015 138
• We expect the solution to the recurrence tobe number of levels × cost of each level:
log ) = lg )• Not every level contributes a cost of• If this recursion tree were a complete binary
tree of height log , there would be2 leaves
• The cost of each leaf is a constant• The total cost of all leaves would then be
) which, since log 2 > 1, is( lg )
10-Sep-15MAT-72006 AADS, Fall 2015 139
9/10/2015
7
• This recursion tree is not a complete binary tree,however, and has fewer than leaves
• As we go down from the root, more and moreinternal nodes are absent
• Consequently, levels toward the bottomcontribute less than to the total cost
• we can use the substitution method to verify thatlg is an upper bound for the solution to
the recurrence
10-Sep-15MAT-72006 AADS, Fall 2015 140
4.5 The master method
Theorem 4.1 (Master theorem)
Let and > 1, and ) a function, )is de ned on nonneg. integers by recurrence
) + ,where we means either or .
) has the following asymptotic bounds:
10-Sep-15MAT-72006 AADS, Fall 2015 141
9/10/2015
8
1. If = ( ) for some constant > 0,then ) =
2. If ) = ,then ) = lg
3. If ) = for some constant > 0,and if ) ) for some constant < 1and all suf ciently large ,
then ) = )
10-Sep-15MAT-72006 AADS, Fall 2015 142
• In each of the three cases, we compare thefunction ) with the function
• The larger of the two determines the solutionto the recurrence– In case 1, the function is the larger,
then the solution is ) =– In case 3, the function ) is the larger, then
the solution is ) =– In case 2, the two functions are the same
size, we multiply by a logarithmic factor, andthe solution is ) = lg
10-Sep-15MAT-72006 AADS, Fall 2015 143
9/10/2015
9
• Be aware of some technicalities:• In case 1, not only must ) be smaller than
, it must be polynomially smaller• ) must be asymptotically smaller than
by a factor of for some > 0• In case 3, not only must ) be larger than
, it also must be polynomially largerand satisfy the “regularity” condition that
( ) ( )10-Sep-15MAT-72006 AADS, Fall 2015 144
Using the master method
) = 9 3)
• We have = 9, = 3, ) = , and thus)
• Since ) = ), where = 1, wecan apply case 1 of the master theorem andconclude that the solution is )
10-Sep-15MAT-72006 AADS, Fall 2015 145
9/10/2015
10
) = (2 3) + 1
• Here = 1, = 3 2, ) = 1, and= 1
• Case 2 applies, since =(1), and thus the solution to the recurrence
is (lg )
10-Sep-15MAT-72006 AADS, Fall 2015 146
( ) = ( 4) + lg
• We have = 3, = 4, lg , and)
• Since ) = ), where 0.2,case 3 applies provided that the regularitycondition holds for )
• For suf ciently large , we have that) = 3( 4) lg( 4)
(3 4) lg = ) for = 3 4• By case 3, the solution is lg )
10-Sep-15MAT-72006 AADS, Fall 2015 147
9/10/2015
11
• The master method does not apply to) = 2 2) lg
even if it seems to have the proper form:= 2, = 2, lg , and
• One might think that case 3 applies, sinceis asymptotically larger than
• It is, however, not polynomially larger: ratio= lg is asymptotically less than
for any positive constant• Consequently, the recurrence falls into the
gap between case 2 and case 310-Sep-15MAT-72006 AADS, Fall 2015 148
• The recurrence of the running time of thesimple divide-and-conquer algorithm formatrix multiplication
) = 8 2) )
• Now = 8, = 2, and and so
• Since is polynomially larger than– i.e., for = 1case 1 applies, and ( ) =
10-Sep-15MAT-72006 AADS, Fall 2015 149
9/10/2015
12
• The recurrence of the running time ofStrassen’s algorithm
) = 7 2) )
• Here we have = 7, = 2, andand so
• Recalling that 2.80 < lg 7 < 2.81, we see that= O( ) for = 0.8
• Again, case 1 applies, and we have thesolution ( ) =
10-Sep-15MAT-72006 AADS, Fall 2015 150
5 Probabilistic Analysis andRandomized Algorithms
• We need to hire a new of ce assistant with thehelp of an employment agency that sends us acandidate each day
• We interview that person and then decide eitherto hire her or not
• We pay the agency a fee for each interview• To actually hire an applicant is more costly
– We must re the current of ce assistant and– pay a substantial hiring fee to the agency
10-Sep-15MAT-72006 AADS, Fall 2015 151
9/10/2015
13
• We are committed to having, at all times, thebest possible person for the job
• After interviewing an applicant, if she is betterquali ed than the current assistant, we will
re the current of ce assistant and hire thenew applicant
• We are willing to pay the price of this strategy,but wish to estimate what that price will be
10-Sep-15MAT-72006 AADS, Fall 2015 152
5.1 The hiring problem
HIRE-ASSISTANT )
1. best 0 // candidate 0 is a least-quali ed dummy candidate
2. for 1 to3. interview candidate4. if candidate is better than best5. best6. hire candidate
10-Sep-15MAT-72006 AADS, Fall 2015 153
9/10/2015
14
• Interviewing has a low cost, , whereas hiring isexpensive, costing
• Letting be the number of people hired, thetotal cost of the algorithm is )
• No matter how many people we hire, we alwaysinterview candidates and thus always incur thecost
• Let us concentrate on analyzing , the hiringcost
• This quantity varies with each run of thealgorithm
10-Sep-15MAT-72006 AADS, Fall 2015 154
Worst-case analysis
• In the worst case, we hire every candidate thatwe interview– candidates come in increasing order of quality– we hire times, for a total hiring cost of
• We have no idea about the order in which thecandidates arrive, nor do we have any controlover this order
• It is natural to ask what we expect to happen in atypical or average case
10-Sep-15MAT-72006 AADS, Fall 2015 155
9/10/2015
15
Probabilistic analysis
• We can assume that the applicants come in arandom order
• Assume that we can compare any twocandidates and decide the better quali ed
there is a total order on the candidates• Rank each candidate with a unique number
1, … ,• Use ) to denote the rank of applicant
10-Sep-15MAT-72006 AADS, Fall 2015 156
• A higher rank corresponds to a better quali edapplicant
• Ordered list 1 , 2 , … , ) is apermutation of the list 1,2, … ,
• “The applicants come in a random order”– “this list of ranks is equally likely to be any
one of the ! permutations of the numbers 1through ”
• Alternatively, the ranks form a uniform randompermutation; i.e., each of the possible !permutations appears with equal probability
10-Sep-15MAT-72006 AADS, Fall 2015 157
9/10/2015
16
Randomized algorithms
• We call an algorithm randomized if its behavioris determined also by values produced by arandom-number generator RANDOM
• A call to RANDOM returns an integerbetween and , inclusive, with each suchinteger being equally likely
• E.g., RANDOM(0,1) produces 0 with probability1 2, and 1 with probability 1 2
10-Sep-15MAT-72006 AADS, Fall 2015 158
• A call to RANDOM 3,7 returns either 3,4,5,6,7,each with probability 1 5
• Each integer returned by RANDOM isindependent of the integers returned on previouscalls
• In analyzing the running time of a randomizedalgorithm, we take the expectation of the runningtime over the distribution of values returned bythe RANDOM
• We refer to the running time of a randomizedalgorithm as an expected running time
10-Sep-15MAT-72006 AADS, Fall 2015 159
9/10/2015
17
5.2 Indicator random variables
• Indicator random variables provide a convenientmethod for converting between probabilities andexpectations
• Suppose we are given a sample space and anevent
• Then the indicator random variable }associated with is de ned as
1 if occurs0 if does not occur
10-Sep-15MAT-72006 AADS, Fall 2015 160
• Let us determine the expected number of headsthat we obtain when ipping a fair coin
• Our sample space is = { }, withPr = Pr = 1 2
• We de ne an indicator random variable ,associated with the coin coming up heads ( )
• This variable counts the number of headsobtained in this ip
= 1 if H occurs0 if T occurs
10-Sep-15MAT-72006 AADS, Fall 2015 161
9/10/2015
18
• Recall that E = Pr }• The expected number of heads obtained in one
ip of the coin is simply the expected value ofour indicator variable
E = E= 1 Pr + 0 Pr= 1 (1 2) + 0 (1 2)= 1 2
• Thus the expected number of heads obtained byone ip of a fair coin is 1 2
10-Sep-15MAT-72006 AADS, Fall 2015 162
• The expected value of an indicator randomvariable associated with an event is equal tothe probability that occurs
Lemma 5.1 Given a sample space and anevent in , let }. Then E = Pr{ }
Proof By the de nition of an indicator and thede nition of expected value, we have =
= 1 Pr + 0 Pr = Pr , wheredenotes , the complement of
10-Sep-15MAT-72006 AADS, Fall 2015 163
9/10/2015
19
• Compute the number of heads in coin ips• Let be the indicator associated with the event
in which the th ip comes up heads:{the th flip results in the event }
• Let denote the total number of heads in thecoin ips, so that
=
• To compute the expected number of heads, takethe expectation of both sides to obtain
E = E
10-Sep-15MAT-72006 AADS, Fall 2015 164
• Linearity of expectationE = E + E
makes the use of indicator random variables apowerful analytical technique– it applies even when there is dependence
among the variables• We now can easily compute the expected
number of heads:
E = E = E
1 2 = 2
10-Sep-15MAT-72006 AADS, Fall 2015 165
9/10/2015
20
Analysis of the hiring problem
• Let be the random variable whose valuenumber of times we hire a new assistant
• let be the indicator random variableassociated with the event in which the thcandidate is hired
candidate is hired= 1 if candidate is hired
0 if candidate is not hired
•
10-Sep-15MAT-72006 AADS, Fall 2015 166
• By Lemma 5.1E = Pr candidate is hired
and we must compute the probability that lines5–6 of HIRE-ASSISTANT are executed
• Candidate is hired (line 6) exactly when she isbetter than each of candidates 1, … ,
• We assume that the candidates arrive in arandom order: the rst candidates haveappeared in a random order
• Any one of these rst candidates is equallylikely to be the best-quali ed so far
10-Sep-15MAT-72006 AADS, Fall 2015 167
9/10/2015
21
• Candidate has a probability of 1/ of beingbetter quali ed than candidates 1, … , 1 andthus a probability of 1/ of being hired.
• By Lemma 5.1, we conclude that E = 1/• Now we can compute E :
E = E = E
= = ln (1)
by harmonic series
= = ln (1)
10-Sep-15MAT-72006 AADS, Fall 2015 168
• We interview people, but actually hire onlyapprox. ln of them, on average
• The following lemma summarize this
Lemma 5.2 Assuming that candidates come inrandom order, algorithm HIRE-ASSISTANT has anaverage-case total hiring cost of ln )
Proof Follows immediately from our de nition ofthe hiring cost and the equation, which shows thatthe expected number of hires is approximately ln
10-Sep-15MAT-72006 AADS, Fall 2015 169
9/10/2015
22
5.3 Randomized algorithms
• The algorithm above is deterministic• The number of times we hire a new of ce
assistant differs for different inputs• Given the rank list = 1,2,3,4,5,6,7,8,9,10 , a
new assistant is always hired 10 times• For the list of ranks = 10,9,8,7,6,5,4,3,2,1 , a
new assistant is hired only once• = 5,2,1,8,4,7,10,9,3,6 , a new assistant is
hired three times
10-Sep-15MAT-72006 AADS, Fall 2015 170
• Consider an algorithm that rst permutes theinput before determining the best candidate
• Now we randomize in the algorithm, not in theinput distribution
• For input we cannot say how many times thebest is updated – it differs with each run
• Not even an enemy can produce a bad input,random permutation yields input order irrelevant
• The algorithm performs badly only if the random-number generator produces an “unlucky”permutation
10-Sep-15MAT-72006 AADS, Fall 2015 171
9/10/2015
23
RANDOMIZED-HIRE-ASSISTANT )
1. randomly permute the list of candidates2. best 0 // candidate 0 is a least-quali ed
// dummy candidate3. for 1 to4. interview candidate5. if candidate is better than candidate best6. best7. hire candidate
10-Sep-15MAT-72006 AADS, Fall 2015 172
Randomly permuting arrays
• We assume that we are given an array which,w.l.o.g., contains the elements 1, 2, … ,
• Produce a random permutation of the array• Assign element ] a random priority ], and
sort the elements according to priorities• E.g., if our initial array is = 1,2,3,4 and we
choose random priorities = 36,3,62,19 , wewould produce array = 2,4,1,3
• We call this procedure PERMUTE-BY-SORTING
10-Sep-15MAT-72006 AADS, Fall 2015 173
9/10/2015
24
PERMUTE-BY-SORTING )
1. . length2. let [1. . ] be a new array3. for 1 to4. RANDOM(1, )5. sort , using as sort keys• We use a range of 1 to for random
numbers to make it likely that all the prioritiesin are unique
10-Sep-15MAT-72006 AADS, Fall 2015 174
• It remains to prove that the procedure producesa uniform random permutation,– It is equally likely to produce every
permutation of the numbers 1 through
Lemma 5.4 PERMUTE-BY-SORTING produces auniform random permutation of the input,assuming that all priorities are distinct
Proof See the book.
10-Sep-15MAT-72006 AADS, Fall 2015 175
9/10/2015
25
• It is better to permute the given array in place• RANDOMIZE-IN-PLACE does so in ) time• In its th iteration, it chooses the element ]
randomly from among elements through]
• After the th iteration, is never altered
RANDOMIZE-IN-PLACE )12 for 1 to3 swap with [RANDOM( , )]
10-Sep-15MAT-72006 AADS, Fall 2015 176
• A -permutation on a set of elements is asequence containing of the elements, withno repetitions
• There are ! ! -permutations
loop invariant:• Just prior to the th iteration of the for loop of
lines 2–3, for each possible 1)-permutationof the elements, the subarray [1. . 1]contains this 1)-permutation with probability
+ 1 ! !
10-Sep-15MAT-72006 AADS, Fall 2015 177
9/10/2015
26
Initialization: loop invariant trivially holds
Maintenance:• Consider a particular -permutation, and denote
the elements in it by , … ,• This permutation consists of an 1)-
permutation , … , followed by thevalue that the algorithm places in ]
• Let denote the event in which the rst 1)iterations have created the particular 1)-permutation , … , in [1. . 1]
10-Sep-15MAT-72006 AADS, Fall 2015 178
• By the loop invariant, Pr = + 1 ! !• Let be the event that th iteration puts in
position ]• The -permutation , … , appears in
1. . precisely when both and occurPr Pr Pr{
• Pr = 1 + 1 because in line 3 thealgorithm chooses randomly from the
+ 1 values in positions . .
Pr =1
+ 1+ 1 !!
=!
!10-Sep-15MAT-72006 AADS, Fall 2015 179
9/10/2015
27
Termination:• At termination, + 1, and we have that the
subarray 1. . is a given -permutation withprobability
+ 1 + 1 ! ! = 0! ! = 1 !
• Thus, RANDOMIZE-IN-PLACE produces a uniformrandom permutation
10-Sep-15MAT-72006 AADS, Fall 2015 180