umass lowell computer science 91.404 analysis of algorithms spring, 2002

UMass Lowell Computer Science 91.404

Analysis of Algorithms Spring, 2002

UMass Lowell Computer Science 91.404

Analysis of Algorithms Spring, 2002

Chapter 5 Lecture Chapter 5 Lecture

Randomized AlgorithmsRandomized AlgorithmsSections 5.1 – 5.3Sections 5.1 – 5.3

source: 91.404 textbook Cormen et al.source: 91.404 textbook Cormen et al.

The Hiring ProblemThe Hiring Problem

HIRE-ASSISTANT(HIRE-ASSISTANT(n)n)bestbest 0 0 candidate 0 is a least-qualified dummy candidatecandidate 0 is a least-qualified dummy candidate

forfor ii 1 1 to to nndo do interview candidate interview candidate ii

if if candidate candidate i i is better than candidate is better than candidate bestbestthenthen best best ii hire candidate hire candidate ii

Job candidates are numbered 1…Job candidates are numbered 1…nn

Analysis GoalAnalysis Goal::- Analyze interview & hiring cost instead of running timeAnalyze interview & hiring cost instead of running time- Interviewing Cost = cInterviewing Cost = cii << Hiring Cost = c << Hiring Cost = chh

- Assume m people are hiredAssume m people are hired- Total Cost:Total Cost: )( hi mcncO

Analyzing the Hiring ProblemAnalyzing the Hiring Problem

Worst-Case Analysis:Worst-Case Analysis: Hire every candidate interviewedHire every candidate interviewed How can this occur?How can this occur?

If candidates come in increasing quality orderIf candidates come in increasing quality order Hire n times: total hiring cost = O(ncHire n times: total hiring cost = O(nchh))

Probabilistic Analysis:Probabilistic Analysis: Appropriate if information about random distribution of inputs Appropriate if information about random distribution of inputs

is knownis known Use a random variable to represent cost (or run-time) Use a random variable to represent cost (or run-time) Find expected (average) cost (or run-time) over all inputsFind expected (average) cost (or run-time) over all inputs We’ll use this technique to analyze hiring cost…We’ll use this technique to analyze hiring cost…

First need to introduce indicator random variables to simplify analysisFirst need to introduce indicator random variables to simplify analysis

Indicator Random VariablesIndicator Random Variables

Indicator random variable I{A} associated with event A:Indicator random variable I{A} associated with event A:

ExampleExample: Find expected number of heads when flipping a : Find expected number of heads when flipping a fair coinfair coin Sample space S = {Sample space S = {H, T H, T }} Random variable Random variable Y Y takes on values takes on values H, TH, T

Prob(Prob(H H ) = Prob() = Prob(T T ) = 1/2) = 1/2 XXHH counts number of heads in a counts number of heads in a single single flipflip

Expected number of heads in a single coin flip = expected value of Expected number of heads in a single coin flip = expected value of XXHH

occurnot does if0

occurs if1}{

A

AAI

TY

HYHYIX H if0

if1}{

}]{[][ HYIEXE H }Pr{0}Pr{1 TYHY 2

1

2

10

2

11

Indicator Random Variables(continued)Indicator Random Variables(continued)

Lemma 5.1Lemma 5.1: Given sample space S and an event A : Given sample space S and an event A in S. Let Xin S. Let XAA = I{A}. Then E[X = I{A}. Then E[XAA] = Pr{A}. ] = Pr{A}.

ProofProof: By definitions of indicator random variable : By definitions of indicator random variable and expectation:and expectation:

Useful for analyzing sequence of events…Useful for analyzing sequence of events…

}]{[][ AIEXE A

}Pr{0}Pr{1 AA

}Pr{A

Analyzing the Hiring Problemusing Indicator Random VariablesAnalyzing the Hiring Problemusing Indicator Random Variables

Assume candidates arrive in random orderAssume candidates arrive in random order X = X = random variable modeling number of times we hire random variable modeling number of times we hire

Without indicator Without indicator random variables:random variables:

With indicator With indicator random variables:random variables:

n

x

xXxXE1

)Pr(][difficult difficult calculation calculation in this casein this case

hired is candidate if0

hired is candidate if1}hired is candidate{

noti

iiIX i

nXXXX 21

}hired is candidatePr{][ iXE i by by Lemma 5.1Lemma 5.1Need to find thisNeed to find this

Analyzing the Hiring Problemusing Indicator Random Variables (continued)


Candidate Candidate ii is hired when candidate is hired when candidate ii is better than each of candidates 1… is better than each of candidates 1…ii-1-1 Randomness assumption: any of first Randomness assumption: any of first ii candidates is equally likely to be best- candidates is equally likely to be best-

qualified so farqualified so far Candidate Candidate ii has probability of 1/ has probability of 1/ii of being better than each of candidates 1… of being better than each of candidates 1…ii-1-1

Probability candidate Probability candidate ii will be hired is therefore: 1/ will be hired is therefore: 1/ii

iiXE i /1}hired is candidatePr{][




Need to calculate probability that these lines are executed



iiXE i /1}hired is candidatePr{][




Assuming that

candidates are presented in random order, total

hiring cost is in O(ch ln n)

)1(ln/1][][][111

OniXEXEXEn

i

n

ii

n

ii

nXXXX 21

by by Linearity of Linearity of ExpectationExpectation

And now for something different…And now for something different…

Randomized AlgorithmRandomized Algorithm:: Put randomness into algorithm itselfPut randomness into algorithm itself Make choices randomly (e.g. using coin flips)Make choices randomly (e.g. using coin flips)

Use pseudorandom-number generator Use pseudorandom-number generator Algorithm itselfAlgorithm itself behaves randomly behaves randomly Different fromDifferent from using probabilistic assumptions about using probabilistic assumptions about

inputsinputs to to analyzeanalyze average-case behavior of a average-case behavior of a deterministic (non-random) algorithmdeterministic (non-random) algorithm

Randomized algorithms are often easy to design & Randomized algorithms are often easy to design & implement & can be very useful in practiceimplement & can be very useful in practice

HH

Randomized Hiring AlgorithmRandomized Hiring Algorithm

RANDOMIZED-HIRE-ASSISTANT(RANDOMIZED-HIRE-ASSISTANT(n)n)randomly permute the list of candidatesrandomly permute the list of candidatesbestbest 0 0 candidate 0 is a least-qualified dummy candidatecandidate 0 is a least-qualified dummy candidate



deterministic

random

)(,),2(),1( nrankrankrank Represent an input using candidate ranks:Represent an input using candidate ranks:

Randomly permuting candidate list creates a random list of candidate ranks.Randomly permuting candidate list creates a random list of candidate ranks.

The expected cost of RANDOMIZED-HIRE-ASSISTANT is in O(ch ln n).

Note: We no longer need to assume that candidates are presented in random order!

Nothing is ever really free…Nothing is ever really free…

RANDOMIZED-HIRE-ASSISTANT(RANDOMIZED-HIRE-ASSISTANT(n)n)randomly permute the list of candidatesrandomly permute the list of candidatesbestbest 0 0 candidate 0 is a least-qualified dummy candidatecandidate 0 is a least-qualified dummy candidate



How do we do this well?How much time does it

take?

)(,),2(),1( nrankrankrankA GoalGoal: permute array A : permute array A

ApproachApproach: assign each element A[ i ] a : assign each element A[ i ] a random priorityrandom priority P[ i ] P[ i ]

Randomized Permutation AlgorithmRandomized Permutation Algorithm

PERMUTE-BY-SORTING(APERMUTE-BY-SORTING(A))nn lengthlength[A][A]forfor ii 1 1 to to nn

do do P[ P[ i i ] RANDOM(1,] RANDOM(1,nn33))sort A, using P as sort keyssort A, using P as sort keysreturnreturn A A

ApproachApproach: assign each element A[ i ] a : assign each element A[ i ] a random priorityrandom priority P[ i ] P[ i ]

Choose a random number between 1

and n3

(makes it more likely that all priorities are

unique)This step dominates the running time. It takes (n lg n) time if pairs of

values are compared while sorting

Now we need to show that PERMUTE-BY-SORTING Now we need to show that PERMUTE-BY-SORTING produces a uniform random permutation of the input, produces a uniform random permutation of the input,

assuming all priorities are distinct…assuming all priorities are distinct…

Randomized Permutation Algorithm AnalysisRandomized Permutation Algorithm Analysis

ClaimClaim: PERMUTE-BY-SORTING produces a uniform random : PERMUTE-BY-SORTING produces a uniform random permutation of the input, assuming all priorities are distinct.permutation of the input, assuming all priorities are distinct.

}Pr{ 1321 nn XXXXX

Proof SketchProof Sketch: Consider permutation in which each A[ : Consider permutation in which each A[ i i ] has ] has i i th smallest th smallest prioritypriorityTo show this permutation occurs with probability 1/n! …To show this permutation occurs with probability 1/n! …

Let Let XXii be event that element A[ i ] receives i th smallest priority.be event that element A[ i ] receives i th smallest priority.

Probability that, for all i, event Probability that, for all i, event XXii occurs is : occurs is :

}|Pr{}|Pr{}|Pr{}Pr{ 11123121 XXXXXXXXX nn

To complete the proof, now apply the argument To complete the proof, now apply the argument above to any fixed permutation of {1,2,…,n}:above to any fixed permutation of {1,2,…,n}: )(,),2(),1( n

}Pr{ 1321 nn XXXXX

!

1

1

1

2

1

1

11

nnn

Randomized Permutation Algorithm ImprovementRandomized Permutation Algorithm Improvement

RANDOMIZE-IN-PLACE(ARANDOMIZE-IN-PLACE(A))nn lengthlength[A][A]forfor ii 1 1 to to nn

do swap do swap A[ A[ i i ] A[RANDOM(] A[RANDOM(ii,,nn)])]

ClaimClaim: RANDOMIZE-IN-PLACE produces a uniform random : RANDOMIZE-IN-PLACE produces a uniform random permutation of the input.permutation of the input.

Proof uses a loop invariant for the Proof uses a loop invariant for the forfor loop. See p. 103 of text for detais. loop. See p. 103 of text for detais.

umass lowell computer science 91.404 analysis of algorithms spring, 2002

Documents