randomized algorithms - stanford cs theory

Randomized Algorithms

Prabhakar RaghavanIBM Almaden Research Center

San Jose� CA�

� Typeset by FoilTEX �

Deterministic Algorithms

ALGORITHMINPUT

OUTPUT

Goal� To prove that thealgorithm solves the problemcorrectly �always� and quickly

�typically� the number of stepsshould be polynomial in thesize of the input��

� Typeset by FoilTEX � �


ALGORITHMINPUT

OUTPUT

RANDOM NUMBERS

� In addition to input� algorithmtakes a source of randomnumbers and makes randomchoices during execution�

� Behavior can vary even on a�xed input�



ALGORITHMINPUT

OUTPUT

RANDOM NUMBERS

� Design algorithm � analysisto show that this behavior islikely to be good� on every

input�

�The likelihood is over therandom numbers only��


Not to be confused with the

Probabilistic Analysis of

Algorithms

ALGORITHMINPUT

OUTPUT

RANDOM

DISTRIBUTION

� Here the input is assumedto be from a probabilitydistribution�

� Show that the algorithmworksfor most inputs�


Monte Carlo and Las Vegas

A Monte Carlo algorithm runsfor a �xed number of steps�and produces an answer that iscorrect with probability � ��

A Las Vegas algorithm alwaysproduces the correct answer�its running time is a randomvariable whose expectation isbounded �say by a polynomial��


Monte Carlo and Las Vegas

These probabilitiesexpectationsare only over the randomchoices made by the algorithm independent of the input�

Thus independent repetitionsof Monte Carlo algorithms drivedown the failure probabilityexponentially�


Advantages of randomized

algorithms

� Simplicity

� Performance

For many problems� a randomizedalgorithm is the simplest� thefastest� or both�


Scope

� Number�theoretic algorithms�Primality testing �Monte Carlo��

� Data structures� Sorting�order statistics� searching�computational geometry�

� Algebraic identities� Polynomialand matrix identity veri�cation�Interactive proof systems�


� Mathematical programming�Faster algorithms for linearprogramming� Rounding linearprogram solutions to integerprogram solutions�

� Graph algorithms� Minimumspanning trees� shortest paths�minimum cuts�

� Counting and enumeration�Matrix permanent� Countingcombinatorial structures�


� Parallel and distributed computing�Deadlock avoidance� distributedconsensus�

� Probabilistic existence proofs�Show that a combinatorialobject arises with non�zeroprobability among objects drawnfrom a suitable probabilityspace�

� Typeset by FoilTEX � ��

� Derandomization� First devisea randomized algorithm� thenargue that it can be �derandomized to yield a deterministic algorithm�


Game�tree evaluation

MAX

MIN

MAX MAX

MIN

MAX MAX

1 7 9 5 3 2 17 12

7 9 3 17

9 3

9

� Tree with alternating MAXMINnodes�Each leaf has a real value�

� Goal� To evaluate the root�


Game�tree evaluation

MAX

MIN

MAX MAX

MIN

MAX MAX

1 7 9 5 3 2 17 12

7 9 3 17

9 3

9

� Cost� The number of leavesinspected�

� Algorithm� Computes leaf�inspection sequence�


Simple special case

� Uniform binary tree� height h�with n � �h leaves�

� All values are boolean� soMAX�OR and MIN�AND�

Exercise� Every deterministicalgorithm can be forced to readn leaves�


Randomized algorithm

AND

OR OR

AND AND AND AND

1 0

1 0 1 1 1 0 1 0

0 1 0 0

Evaluate �recursively� a randomchild of the current node�

If this does not determinethe value of the current node�evaluate the other child�


Analysis of tree evaluation

AND�OR trees of depth �k �klevels of AND and k levels ofOR on any path�� will prove byinduction on k�

Expected cost for an ORsubtree evaluating to ��

�

�� k��

�� k��

�� k��


Analysis of tree evaluation

For an OR subtree evaluatingto �� we must evaluate bothof its subtrees� incurring cost��k��

Now consider the root AND��a� If it is �� both OR subtreesmust evaluate to �� expectedcost paid is twice above� � �k��b� If it is �� at least one ORsubtree evaluates to ��


Game tree analysis

Expected cost for root ANDevaluating to ��

� � �k��

�

�

��k�� k�

Number of leaves in tree � k�

Thus expected cost of randomizedalgorithm � n��


Lower bounds and the

minimax principle

Basic Idea� For a lowerbound on the performance ofall randomized algorithms for aproblem� derive instead a lowerbound for any deterministicalgorithm for the problem when

the inputs are drawn from a

probability distribution �of yourchoice��


Let I denote the set ofinstances of the problem� andA the set of deterministicalgorithms for the problem�Any randomized algorithm canbe viewed as a probabilitydistribution on the algorithms inA�


Minimax Principle

For all distributions p over Iand q over A�minA�A

EC�Ip� A�� maxI�I

EC�I� Aq��


Lower bound for game tree

evaluation

We will specify a probabilitydistribution on instances ��values for the leaves� and lowerbound the expected runningtime of any deterministic algorithm�


NOR trees instead

Exercise� Show that a balancedAND�OR tree of even depth isequivalent to a tree of the samedepth� all of whose nodes areNOR nodes�

We will show the lower boundfor a NOR tree with n leaves�


The input distribution

Let p � ��p��

Claim� The value of anynode is � with probability p�independent of all other nodesat the same level�

Claim� Any deterministicalgorithmmay as well determinethe value of one sub�tree of anode� before inspecting any leafof its sibling sub�tree�


The Analysis

Let W �h� be the expectednumber of leaves it inspects indetermining the value of a nodeat distance h from the leaves�

Clearly

W �h� � W �h��p�W �h��

Letting h � log� n� this gives alower bound of n�� Typeset by FoilTEX � ��

Exercise� Why is this lowerbound weak�


The ��SAT Problem

Given a set of boolean clausesin CNF each containing twoliterals� �nd a satisfying assignmentif one exists�

�x� � x�� x� � x��x� � x��

� Start with any tentative assignment

� If there is an unsatis�edclause� pick one of its twoliterals at random� and �ip it�


� If no solution found in �n�

steps� declare �none exists �

Monte Carlo� If a solutionexists� will �nd it with probability� ��If not� will always declare �noneexists �


Random Walk Analysis

What is the probability ofmissing a satisfying assignmentin �n� steps�

� Fix any particular satisfyingassignment A�

� Consider the number of literalson which the algorithm�stentative assignment agreeswith A�


� This is a random walk on theintegers that increases withprobability at least �� ateach step�

Expected time to �nd anassignment � n��Markov�s inequality� probabilityof missing an assignment in �n�

steps is � ��


Binary planar partitions

L1

L2 L3s3

1

s2

L1L2

L3

s

1s s3 s3 s2

Given n non�intersecting line�segments in the plane� build asmall linear decision tree thathas �pieces of� at most onesegment in each cell�


Autopartitions

Choose the input segments inrandom order�Extend the chosen segment �ifnecessary� to cut any cellswhere it is not �alone �

The expected size of theresulting tree is � n� �nHn�


Analysis of autopartition size

u

v1

v2

De�ne l�u� to be the linecontaining segment u�

u a v denotes l�u� cuts v inthe constructed tree�

index�u� v� is i if l�u� intersectsi � � other segments beforehitting v�


Autopartitions

Pru a v� � ��index�u� v��

Thus the expected size of thetree constructed is

n�X

u

X

v �u

Pru a v� �

n�X

u

X

v �u

�

index�u� v� � �


For any u� i� index�u� v� � i forat most two v� thus the abovesum is � n� �nHn�

It follows that for any n

segments in the plane� there

always exists an autopartition ofsize O�n log n��


Matrix product veri�cation

Suppose A�B and C are n �n matrices with entries from a�nite �eld F � We wish to verifywhether AB � C�

Multiplying out takes aboutn� steps this can be cut toO�n�� steps using sophisticatedalgorithms�


Simple randomized

algorithm

� Pick an n�vector x withentries randomly drawn fromF �

� Compute z � A�Bx��If z � Cx output AB � C�else output AB � C�

Takes O�n�� steps�


Simple randomized

algorithm

If AB � C� will always outputAB � C�

If AB � C� will outputAB � C with probability atmost ��jFj�


Sources

R�M� Karp� An introduction torandomized algorithms� DiscreteApplied Mathematics� ��

R� Motwani and P� Raghavan�Randomized Algorithms� CambridgeUniversity Press� ��

D� J� A� Welsh� Randomisedalgorithms� Discrete Applied

Mathematics� ��


randomized algorithms - stanford cs theory

Documents