Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Chapter 7: Randomized Computation

Jhoirene Clemente

February 7, 2013

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Outline

1 Introduction

2 Probabilistic Turing Machines (PTM)

3 RP, coRP, ZPP

4 Error Reduction for BPP

5 Relation of BPP with other classes

6 Summary

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

A randomized algorithm is an algorithm that is allowed access toa source of independent, unbiased, random bits. The algorithm isthen permitted to use these random bits to influence itscomputation.

We want to study TMs that has the power to toss random coins.

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

A randomized algorithm is an algorithm that is allowed access toa source of independent, unbiased, random bits. The algorithm isthen permitted to use these random bits to influence itscomputation.

We want to study TMs that has the power to toss random coins.

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Introduction

Recall

Definition (Turing Machine)

A Turing machine is a 7-tuple, (Q,Σ,Γ, δ, q0, qaccept, qreject), where Q,Σ, Γ are all finites sets and

1 Q is the set of states,

2 Σ is the input alphabet not containing the blank symbol t.

3 Γ is the tape alphabet, where t ∈ Γ and Σ ⊆ Γ,

4 δ : Q× Γ→ Q× Γ× {L,R} is the transition function,

5 q0 ∈ Q is the start,

6 qaccept ∈ Q is the accept state, and

7 qreject ∈ Q is the reject state, where qreject 6= qaccept

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Example Deterministic Turing Machine

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Nondeterministic Turing Machine

Definition (Nondeterministic Turing Machine)

A nondeterministic Turing machine is a 7-tuple,(Q,Σ,Γ, δ, q0, qaccept, qreject), where Q, Σ, Γ are all finites sets and

δ : Q× Γ→ P(Q× Γ× {L,R})

From each configuration, there may be several possible transitions,generating a ‘tree’ of computations.If some computation branch leads to accept, then the machine acceptsits input.

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Outline

1 Introduction

2 Probabilistic Turing Machines (PTM)

3 RP, coRP, ZPP

4 Error Reduction for BPP

5 Relation of BPP with other classes

6 Summary

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Probabilistic Turing Machines (PTMs)

Definition (PTMs)

A Probabilistic Turing Machine (PTM) is a Turing machine with twotransition functions δ0, δ1.

To execute a PTM M on an input x, we choose in each step withprobability 1/2 to apply δ0 and with probability 1/2 to apply δ1.This choice is made independently of all previous choices.

The machine outputs ACCEPT (1) or REJECT (0). M(x) denotesthe output of M on x and surely this is a random variable.

For a function T : N→ N,we say that M runs in T(n) time if forany input x, M halts on x within T(|x|) steps regardless of therandom choices M makes.

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Probabilistic Turing Machine (PTM)

In a PTM M, each transition is taken with probability 1/2. If Mhas chosen the transition function t times, then M would havechosen any one of the 2t branches with a probability of 1

2t .We assign a probability to each branch b of M’s computation oninput w as follows. Define the probability of branch b to bePr[b] = 2−k, where k is the number of coin flip steps that occuron branch b.An NDTM accepts if ∃ one accepting branch. Define theprobability that M accepts w to be

Pr[M accepts w] =∑

b is an accepting branch

Pr[b].

The probability that M rejects w is

Pr[M rejects w] = 1− Pr[M accepts w]S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Probabilistic Turing Machine (PTM)

In a PTM M, each transition is taken with probability 1/2. If Mhas chosen the transition function t times, then M would havechosen any one of the 2t branches with a probability of 1

2t .We assign a probability to each branch b of M’s computation oninput w as follows. Define the probability of branch b to bePr[b] = 2−k, where k is the number of coin flip steps that occuron branch b.An NDTM accepts if ∃ one accepting branch. Define theprobability that M accepts w to be

Pr[M accepts w] =∑

b is an accepting branch

Pr[b].

The probability that M rejects w is

Pr[M rejects w] = 1− Pr[M accepts w]S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Probabilistic Turing Machine (PTM)

In a PTM M, each transition is taken with probability 1/2. If Mhas chosen the transition function t times, then M would havechosen any one of the 2t branches with a probability of 1

2t .We assign a probability to each branch b of M’s computation oninput w as follows. Define the probability of branch b to bePr[b] = 2−k, where k is the number of coin flip steps that occuron branch b.An NDTM accepts if ∃ one accepting branch. Define theprobability that M accepts w to be

Pr[M accepts w] =∑

b is an accepting branch

Pr[b].

The probability that M rejects w is

Pr[M rejects w] = 1− Pr[M accepts w]S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Probabilistic Turing Machine (PTM)

In a PTM M, each transition is taken with probability 1/2. If Mhas chosen the transition function t times, then M would havechosen any one of the 2t branches with a probability of 1

2t .We assign a probability to each branch b of M’s computation oninput w as follows. Define the probability of branch b to bePr[b] = 2−k, where k is the number of coin flip steps that occuron branch b.An NDTM accepts if ∃ one accepting branch. Define theprobability that M accepts w to be

Pr[M accepts w] =∑

b is an accepting branch

Pr[b].

The probability that M rejects w is

Pr[M rejects w] = 1− Pr[M accepts w]S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Error Probability

When a PTM recognized a language it must accept all strings in thelanguage and reject all strings out of the language as usual, except thatnow we allow the machine a small probability of error.For 0 ≤ ε < 1/2 we say that M recognizes A with error probability εif

1 w ∈ A implies Pr[M accepts w] ≥ (1− ε), and

2 w /∈ A implies Pr[M rejects w] ≥ (1− ε).

Definition (Bounded Error Probabilistic Polynomial (BPP))

BPP is the class of languages that are recognized by probabilisticpolynomial time Turing machines with an error probability of 1/3.

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Error Probability

When a PTM recognized a language it must accept all strings in thelanguage and reject all strings out of the language as usual, except thatnow we allow the machine a small probability of error.For 0 ≤ ε < 1/2 we say that M recognizes A with error probability εif

1 w ∈ A implies Pr[M accepts w] ≥ (1− ε), and

2 w /∈ A implies Pr[M rejects w] ≥ (1− ε).

Definition (Bounded Error Probabilistic Polynomial (BPP))

BPP is the class of languages that are recognized by probabilisticpolynomial time Turing machines with an error probability of 1/3.

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Error Probability

When a PTM recognized a language it must accept all strings in thelanguage and reject all strings out of the language as usual, except thatnow we allow the machine a small probability of error.For 0 ≤ ε < 1/2 we say that M recognizes A with error probability εif

1 w ∈ A implies Pr[M accepts w] ≥ (1− ε), and

2 w /∈ A implies Pr[M rejects w] ≥ (1− ε).

Definition (Bounded Error Probabilistic Polynomial (BPP))

BPP is the class of languages that are recognized by probabilisticpolynomial time Turing machines with an error probability of 1/3.

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

BPTIME and BPP

Definition (The classes BPTIME and BPP)

For T : N→ N and L ⊆ {0, 1}∗, we say that a PTM M decides L intime T(n), if for every x ∈ {0, 1}∗, M halts in T(|x|) steps regardlessof its random choices, and

Pr[M(x) = L(x)] ≥ 2/3, s

where we denote L(x) = 1 if X /∈ L and L(x) = 0 if x ∈ L.We let BPTIME(T(n)) denote the class of languages decided byPTMs in O(T(n)) time and let

BPP = ∪cBPTIME(nc)

.

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Some Remarks

1 For every input x, M(x) will output the right value L(x) withprobability at least 2/3.

2 The coin need not to be fair. The constant 2/3 is arbitrary in thesense that it can be replaced with any other constant greater than1/2 without changing the classes BPTIME(T(n)) and BPP.

3 Instead of requiring the machine to always halt in a polynomialtime, we could allow it to halt in expected polynomial time.

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Expected Running Time

For a PTM M, and input x, we define the random variable TM,x to bethe running time of M on input x. That is, Pr[TM,x = T] = p if withprobability p over the random choices of M on input x, it will haltwithin T steps.We say that M has an expected running time T(n) if the expectationE[TM,x] is at most T(|x|) for every x ∈ {0, 1}∗.

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Some Remarks

Theorem

P ⊆ BPP ⊆ EXP.

Proof.

Since a deterministic TM is a special case of a PTM (where bothtransition functions are equal) BPP clearly contains P.

Given a polynomial-time PTM M and input x ∈ {0, 1}n in time2poly(n) it is possible to enumerate all possible random choicesand compute precisely the probability that M(x) = 1.

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Example: Primality Testing

In primality testing we are given an integer N and wish to determinewhether or not it is prime.

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Example: Primality Testing

In primality testing we are given an integer N and wish to determinewhether or not it is prime.

PRIME(A, N):IF (gcd(A,N) > 1) or ((N/A) 6= A(N−1)/2 mod N)output COMPOSITE

ELSEoutput PRIME

Theorem

PRIMES ∈ BPP

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Example: Primality Testing

In primality testing we are given an integer N and wish to determinewhether or not it is prime.

PRIME(A, N):IF (gcd(A,N) > 1) or ((N/A) 6= A(N−1)/2 mod N)output COMPOSITE

ELSEoutput PRIME

Theorem

PRIMES ∈ BPP

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Outline

1 Introduction

2 Probabilistic Turing Machines (PTM)

3 RP, coRP, ZPP

4 Error Reduction for BPP

5 Relation of BPP with other classes

6 Summary

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

One sided zero sided error: RP, coRP, ZPP

The probabilistic primality algorithm has one-sided error. When thealgorithm outputs accept, we know that the input must be prime.When the output is reject, we know only that the input could be primeor composite.

Definition (RP)

A language L ⊆ {0, 1}∗ is said to be in RTIME(T (n)) if there exists aPTM running in time T(n) s.t.

∀ x ∈ L, Pr[M(x) = 1] ≥ 2/3

∀ x ∈ L, Pr[M(x) = 0] = 1

The class RP =⋃

c<0 RTIME(T(n)).

Theorem

RP ⊆ BPP

RP ⊆ NP but we do not know whether BPP ⊆ NP.S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

One sided zero sided error: RP, coRP, ZPP

The probabilistic primality algorithm has one-sided error. When thealgorithm outputs accept, we know that the input must be prime.When the output is reject, we know only that the input could be primeor composite.

Definition (RP)

A language L ⊆ {0, 1}∗ is said to be in RTIME(T (n)) if there exists aPTM running in time T(n) s.t.

∀ x ∈ L, Pr[M(x) = 1] ≥ 2/3

∀ x ∈ L, Pr[M(x) = 0] = 1

The class RP =⋃

c<0 RTIME(T(n)).

Theorem

RP ⊆ BPP

RP ⊆ NP but we do not know whether BPP ⊆ NP.S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

coRP

Definition (coRP)

A language L ⊂ {0, 1}∗ is said to be in coRP if there exists a PTMrunning in polynomial time s.t.

∀ x ∈ L, Pr[M(x) = 1] = 1

∀ x ∈ L, Pr[M(x) = 0] ≥ 2/3

Theorem

PRIMES ∈ coRP

Theorem

coRP ⊆ BPP

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

ZPP

Definition (ZTIME(T(n)) and ZPP)

The class ZTIME(T(n)) contains all languages L for which there is anexpected time O(T((n)) machine that never errs. That is

x ∈ L⇒ Pr[M accepts x] =1

x /∈ L⇒ Pr[M halts without accepting x] =1

We define ZPP =⋃

c>0 ZTIME(nc).

Theorem

ZPP = RP ∩ coRP

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Complexity Classes

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Outline

1 Introduction

2 Probabilistic Turing Machines (PTM)

3 RP, coRP, ZPP

4 Error Reduction for BPP

5 Relation of BPP with other classes

6 Summary

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Amplification Lemma

We defined BPP with an error probability of 1/3, but any constanterror probability would yield an equivalent definition as long as it isstrictly between 0 and 1/2 by virtue of amplification lemma.

Lemma (Amplification Lemma)

Let ε be a fixed constant strictly between 0 and 1/2. Then for anypolynomial poly(n) with a probabilistic polynomial time Turingmachine M1 that operates with error probability ε has an equivalentprobabilistic polynomial time Turing machine M2 that operates withan error probability of 2−poly(n).

PROOF IDEA: M2 simulates M1 by running it a polynomial numberof times and taking the majority vote of the outcomes. The probabilityof error decreases exponentially with the number of runs of M1 made.

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Outline

1 Introduction

2 Probabilistic Turing Machines (PTM)

3 RP, coRP, ZPP

4 Error Reduction for BPP

5 Relation of BPP with other classes

6 Summary

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Relation of BPP with other classes

1 P ⊆ BPP

2 BPP ⊆ EXP

3 RP ⊆ BPP

4 coRP ⊆ BPP

5 ZPP = RP ∩ coRP

6 ZPP ⊆ BPP

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

BPP is in P/poly

Theorem

BPP ⊆ P/poly

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

PH

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Is BPP in PH?

It is enough to prove that BPP ⊆ Σp2 because BPP is closed under

complementation, i.e. BPP = coBPP.

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Outline

1 Introduction

2 Probabilistic Turing Machines (PTM)

3 RP, coRP, ZPP

4 Error Reduction for BPP

5 Relation of BPP with other classes

6 Summary

S. Arora, B. Barak

Computational Complexity: A Modern Approach

Introduction Probabilistic Turing Machines (PTM) RP, coRP, ZPP Error Reduction for BPP Relation of BPP with other classes Summary

Summary

1 The class BPP consists of languages that can be solved by aprobabilistic polynomial-time algorithm.

2 RP, coRP, and ZPP are subclasses of BPP corresponding toprobabilistic algorithms with one sided and ”zero sided” error.

3 Using repetition, we can considerably amplify the successprobability of probabilistic algorithms.

4 We only know that P ⊆ BPP ⊆ EXP, but we suspect thatBPP = P.

5 BPP is a subset of both P/poly and PH. in particular, the latterimplies that if NP = P then BPP = P.

S. Arora, B. Barak

Computational Complexity: A Modern Approach