probability and randomized algorithms

8/6/2019 Probability and Randomized Algorithms

1/14

Probability and

Randomized AlgorithmsProf. Zeph Grunschlag


2/14


3/14

3

Joint and Conditional

Probability Let X, Y be random variables over the resp.

setsX, Y. (Note,X, Y may/may not be same)

DEF: Joint probabilityPr[x,y] is theprobability that (X,Y) = (x,y). (Probability ofboth occurring simultaneously)

DEF: Conditional probabilityis definedby Pr[x|y] = Pr[x,y] / Pr[y] - assuming thatPr[y] > 0.


4/14

4

Independent Variables Random variables are independent if their

probabilities dont depend on each others

values:DEF: X and Y are independent ifPr[x,y] = Pr[x]Pr[y] for allx, y.

LEMMA: Equivalently, X and Y areindependent if (excluding 0-prob.y)x X,y Y,Pr[x|y] = Pr[x]


5/14

5

Bayes Theorem

THM: If Pr[y] > 0 then

Pr[x|y] = Pr[y|x] Pr[x] / Pr[y]


6/14

6

Binomial Rand. Var.DEF: The product of random variables X, Y isthe random variable XY defined onXYwith distribution Pr[(x,y)] = Pr[x]Pr[y].

Assume X a random variable on {0,1} andlet p = Pr[X=1], q = Pr[X=0]

Repeat experiment n times. I.e., take nindependent copies:

result called Binomial random variableBernoullis Thm:

X1X2 Xn

Pr

n

!i=1

Xi= k

=

n

k

pkqnk


7/14

7

Expectation

The average value taken on by a function fon probability distribution X

DEF: The expectation offis defined by:

THM:

COR: For n repetitions of a Binomial randomvariable X consider sum S which counts thenumber outcomes = 1. Then E(S) = np

E(f) = !xX

f(x)px

E(f+g) = E(f) +E(g)


8/14

8

Chernoff Bound

Estimates probability that sum of Binomialexperiment deviate from expected sum np

THM:

Note: probability that sum too big falls offexponentially with n

Pr

S (1+!)pn

e

!2

3pn


9/14

9

Randomized AlgorithmsEquivalent formulations:

Turing machine with coin flips at everystep of computation

Non-deterministic Turing machine withprobability distribution over computationbranches

Nomenclature (varies from author to author):

Monte-Carlo:

Colloquially any randomized algorithm Complexity theory: NOs always right Las-Vegas: always correct, but may fail BPP: answers correct most of the time


10/14

10

Monte Carlo Algorithm

False negative allowed, but no false positivesDEF: A poly-time Monte Carlo algorithmfor the decision problem Pis a poly-time non-

deterministic Turing machine (NDTM) s.t.

Probability measured over coin-flips in TMor equivalently, by taking the ratio ofaccepting branches in NTM to total number

Defines complexity class RP Rand-Poly

Pr[x is accepted] :

1

2x P

= 0 x P


11/14

11

Las Vegas Algorithm

Symmetric version of Monte Carlo - no falsenegatives nor false positives but can fail

DEF: A poly-time Las Vegas algorithm is

a poly-time NDTM with a constant >0 forwhich Pr[fail] for all inputs.

Repeat algorithm to make arbitrarily small

Gives class ZPP Zero-Prob-of-error-Poly ZPP= RP co-RP


12/14

12

Class BPP

BPP = Bounded-Prob-of-error-Poly Most general class - allow false negatives and

positives. Compensate by insisting answercorrect significantly more than half the time

DEF: A poly-time randomized algorithm forthe decision problem Pis a poly-time NDTMwith a constant >0 for which

Chernoff bound implies may assume =0.25

Pr[x is accepted] : 12 + ! x P 1

2 ! x P


13/14

13

Pseudo Random

Sequence

DEF: A pseudo random sequence is adeterministic algorithm from finite bitstringsto infinite bitstrings whose outputs cannot bedistinguished from a random strings by any

BPP algorithm.


14/14

14

-bias Detector

Given: A black box fwhich is known a-priori to have some built-in bias in anunknown direction.

Decide: Which direction the bias is in.n =

x = output of length n fromf

c = number of 1s in x

return (c > n/2) // YES if1-bias, NO if 0-bias

Pr[output is correct] > 3/4 thereforethis problem is in BPP so -biassequences are notpseudorandom.

2

(12

!)!2

probability and randomized algorithms

Documents