fall 2013 cmu cs 15-855 computational complexity lecture 7 alternating quantifiers, ph, pspace....

Fall 2013 CMU CS 15-855Computational Complexity

Lecture 7

Alternating Quantifiers, PH, PSPACE.

These slides are mostly a resequencing of Chris Umans’ slides From CS 151: Complexity Theory (Spring 2013) at Caltech.

10/1/2013

http://users.cms.caltech.edu/~umans/cs151/




10/1/2013 2

A motivating question

• Central problem in logic synthesis:

• Complexity of this problem?– NP-hard? in NP? in coNP? in PSPACE?– complete for any of these classes?

x1 x2

x3 … xn

• given Boolean circuit C, integer k

• is there a circuit C’ of size at most k that computes the same function C does?

10/1/2013 3

Oracle Turing Machines

• Oracle Turing Machine (OTM):– multitape TM M with special “query” tape– special states q?, qyes, qno

– on input x, with oracle language A– MA runs as usual, except…– when MA enters state q?:

• y = contents of query tape• y A transition to qyes

• y A transition to qno

10/1/2013 4


• Nondeterministic OTM – defined in the same way– (transition relation, rather than function)

• oracle is like a subroutine, or function in your favorite programming language – but each call counts as single step

e.g.: given φ1, φ2, …, φn are even # satisfiable?

– poly-time OTM solves with SAT oracle

10/1/2013 5


Shorthand #1: • applying oracles to entire complexity

classes:– complexity class C– language ACA = {L decided by OTM M with oracle A with M “in” C}

– example: PSAT

10/1/2013 6


Shorthand #2:• using complexity classes as oracles:

– OTM M– complexity class C– MC decides language L if for some language A

C, MA decides L

Both together: CD = languages decided by OTM “in” C with oracle language from D

exercise: show PSAT = PNP

10/1/2013 7

The Polynomial-Time Hierarchy

• can define lots of complexity classes using oracles

• the classes on the next slide stand out– they have natural complete problems– they have a natural interpretation in terms of

alternating quantifiers– they help us state certain consequences and

containments (more later)

10/1/2013 8


Σ0 = Π0 = P

Δ1=PP Σ1=NP Π1=coNP

Δ2=PNP Σ2=NPNP Π2=coNPNP

Δi+1=PΣi Σi+i=NPΣi

Πi+1=coNPΣi

Polynomial Hierarchy PH = i Σi

10/1/2013 9


Σ0 = Π0 = P


Πi+1=coNPΣi

• Example:– MIN CIRCUIT: given Boolean circuit C,

integer k; is there a circuit C’ of size at most k that computes the same function C does?

– MIN CIRCUIT Σ2

10/1/2013 10


Σ0 = Π0 = P


Πi+1=coNPΣi

• Example:– EXACT TSP: given a weighted graph G, and

an integer k; is the k-th bit of the length of the shortest TSP tour in G a 1?

– EXACT TSP Δ2

10/1/2013 11

The PH

PSPACE: generalized geography, 2-person games…

3rd level: V-C dimension…

2nd level: MIN CIRCUIT, BPP…

1st level: SAT, UNSAT, factoring, etc…

P

NP coNP

Σ3 Π3

Δ3

PSPACE

EXP

PH

Σ2 Π2

Δ2

10/1//2013

Useful characterization

• Recall: L NP iff expressible asL = { x | 9 y, |y| ≤ |x|k, (x, y) R }

where R P.

• Corollary: L coNP iff expressible asL = { x | 8 y, |y| ≤ |x|k, (x, y) R }

where R P.

10/1//2013

Useful characterization

Theorem: L Σi iff expressible as

L = { x | 9 y, |y| ≤ |x|k, (x, y) R }

where R Πi-1.

• Corollary: L Πi iff expressible as

L = { x | 8 y, |y| ≤ |x|k, (x, y) R }

where R Σi-1.

10/1//2013

Alternating quantifiers

Nicer, more usable version:• LΣi iff expressible as

L = { x | 9y1 8y2 9y3 …Qyi (x, y1,y2,…,yi)R }

where Q= 8/9 if i even/odd, and RP

• LΠi iff expressible as

L = { x | 8y1 9y2 8y3 …Qyi (x, y1,y2,…,yi)R }

where Q= 9/8 if i even/odd, and RP

10/1//2013


• Proof:– ( ) induction on i– base case: true for Σ1=NP and Π1=coNP

– consider LΣi:

L = {x | 9y1 (x, y1) R’ }, for R’ Πi-1

L = {x | 9y1 8y2 9y3 …Qyi ((x, y1), y2,…,yi)R}

L = {x | 9y1 8y2 9y3 …Qyi (x, y1,y2,…,yi)R}

– same argument for L Πi

– ( ) exercise.

10/1//2013


Pleasing viewpoint:

P

NP

coNP

Δ3

Σ2 Π2

Δ2

“9” “8”

“98”

“89”

Σ3 Π3

PSPACEPH

“989”

“898”

“9898989…”

Σi Πi“989…” “898…”

const. # of alternations poly(n)

alternations

10/1//2013

Complete problems

• three variants of SAT:– QSATi (i odd) =

{3-CNFs φ(x1, x2, …, xi) for which

9x18x29x3 … 9xi φ(x1, x2, …, xi) = 1}– QSATi (i even) =

{3-DNFs φ(x1, x2, …, xi) for which

9x18x29x3 … 8xi φ(x1, x2, …, xi) = 1}– QSAT = {3-CNFs φ for which

9x18x29x3 … Qxn φ(x1, x2, …, xn) = 1}

10/1//2013

QSATi is Σi-complete

Theorem: QSATi is Σi-complete.

• Proof: (clearly in Σi)– assume i odd; given L Σi in form

{ x | 9y18y29y3 … 9yi (x, y1,y2,…,yi) R }

…x… …y1… …y2… …y3… … …yi… C

CVAL reduction for R

1 iff (x, y1,y2,…,yi) R

10/1//2013


– Problem set: can construct 3-CNF φ from C:

9z φ(x,y1,…,yi, z) = 1 C(x,y1,…,yi) = 1

– we get:

9y18y2…9yi 9z φ(x,y1,…,yi, z) = 1

Û 9y18y2…9yi C(x,y1,…,yi) = 1 x L

…x… …y1… …y2… …y3… … …yi… C


1 iff (x, y1,y2,…,yi) R

10/1//2013


• Proof (continued)– assume i even; given L Σi in form

{ x | 9y18y29y3 … 8yi (x, y1,y2,…,yi) R }

…x… …y1… …y2… …y3… … …yi… C


1 iff (x, y1,y2,…,yi) R

10/1//2013


– Problem set: can construct 3-DNF φ from C:

8z φ(x,y1,…,yi, z) = 1 C(x,y1,…,yi) = 1

– we get:

9y18y2… 8yi8z φ(x,y1,y2,…,yi, z) = 1

9y18y2…8yi C(x,y1,y2,…,yi) = 1 x L

…x… …y1… …y2… …y3… … …yi… C


1 iff (x, y1,y2,…,yi) R

10/1//2013

QSAT is PSPACE-complete

Theorem: QSAT is PSPACE-complete.• Proof: 8x19x28x3 … Qxn φ(x1, x2, …, xn)?

– in PSPACE: 9x18x29x3 … Qxn φ(x1, x2, …, xn)?– “9x1”: for each x1, recursively solve

8x29x3 … Qxn φ(x1, x2, …, xn)?• if encounter “yes”, return “yes”

– “8x1”: for each x1, recursively solve 9x28x3 … Qxn φ(x1, x2, …, xn)?

• if encounter “no”, return “no”– base case: evaluating a 3-CNF expression– poly(n) recursion depth– poly(n) bits of state at each level

10/1//2013


– given TM M deciding L PSPACE; input x– 2nk possible configurations– single START configuration– assume single ACCEPT configuration

– define:REACH(X, Y, i) configuration Y reachable from

configuration X in at most 2i steps.

10/1//2013


REACH(X, Y, i) configuration Y reachable from configuration X in at most 2i steps.

– Goal: produce 3-CNF φ(w1,w2,w3,…,wm) such that

9w18w2…Qwm φ(w1,…,wm)

Û REACH(START, ACCEPT, nk)

10/1//2013


– for i = 0, 1, … nk produce quantified Boolean expressions ψi(A, B, W)

9w18w2… ψi(A, B, W) REACH(A, B, i)

– convert ψnk to 3-CNF φ • add variables V

• 9w18w2… 9V φ(A, B, W, V) REACH(A, B, nk)

– hardwire A = START, B = ACCEPT

9w18w2… 9V φ(W, V) x L

10/1//2013


– ψo(A, B) = true iff • A = B or • A yields B in one step of M

…

…

STEP STEP STEP STEP

config. A

config. B

Boolean expression of size O(nk)

10/1//2013


– Key observation #1: REACH(A, B, i+1)

9Z [REACH(A, Z, i) REACH(Z, B, i)]

– cannot define ψi+1(A, B) to be

9Z [ψi(A, Z) ψi(Z, B)] (why?)

10/1//2013


– Key idea #2: use quantifiers

– couldn’t do ψi+1(A, B) = 9Z [ψi(A, Z) ψi(Z, B)]

– define ψi+1(A, B) to be

9Z8X8Y [((X=AY=Z)(X=ZY=B)) ψi(X, Y)]– ψi(X, Y) is preceded by quantifiers

– move to front (they don’t involve X,Y,Z,A,B)

10/1//2013


ψo(A, B) = true iff A = B or A yields B in 1 step

ψi+1(A, B) =

9Z8X8Y[((X=AY=Z)(X=ZY=B)) ψi(X, Y)]

– |ψ0| = O(nk)– |ψi+1| = O(nk) + |ψi|

– total size of ψnk is O(nk)2 = poly(n)– logspace reduction

10/1//2013

PH collapse

Theorem: if Σi = Πi then for all j > i

Σj = Πj = Δj = Σi

“the polynomial hierarchy collapses

to the i-th level”

• Proof: – sufficient to show Σi = Σi+1

– then Σi+1= Σi = Πi = Πi+1; apply theorem again

P

NP coNP

Σ3 Π3

Δ3

PH

Σ2 Π2

Δ2

10/1//2013

PH collapse

– recall: L Σi+1 iff expressible as

L = { x | 9 y (x, y) R }

where R Πi

– since Πi = Σi, R expressible as

R = { (x,y) | 9 z ((x, y), z) R’ }

where R’ Πi-1

– together: L = { x | 9 (y, z) (x, (y, z)) R’}– conclude L Σi

10/1//2013

Oracles vs. Algorithms

A point to ponder:• given poly-time algorithm for SAT

– can you solve MIN CIRCUIT efficiently?– what other problems? Entire complexity

classes?• given SAT oracle

– same input/output behavior– can you solve MIN CIRCUIT efficiently?

10/1//2013

Natural complete problems

• We now have versions of SAT complete for levels in PH, PSPACE

• Natural complete problems?– PSPACE: games

– PH: almost all natural problems lie in the second and third level

10/1//2013

Natural complete problems in PH

– MIN CIRCUIT• good candidate to be 2-complete, still open

– MIN DNF: given DNF φ, integer k; is there a DNF φ’ of size at most k computing same function φ does?

Theorem (U): MIN DNF is Σ2-complete.

10/1//2013

Natural complete problems in PSPACE

• General phenomenon: many 2-player games are PSPACE-complete.

– 2 players I, II– alternate pick-

ing edges– lose when no

unvisited choice

pasadena

athensauckland

san francisco

oakland

davis

• GEOGRAPHY = {(G, s) : G is a directed graph and player I can win from node s}

10/1//2013


Theorem: GEOGRAPHY is PSPACE-complete.

Proof:– in PSPACE

• easily expressed with alternating quantifiers

– PSPACE-hard• reduction from QSAT

10/1//2013


9x18x29x3…(x1x2x3)(x3x1)…(x1x2)

true false

true false

true false

…

x1

x2

x3

I

II

II

II

II

I

I

I

I

alternately pick truth assignment

pick a clause

I

II

I pick a true literal?

10/1//2013

Karp-Lipton

• we know that P = NP implies SAT has polynomial-size circuits.– (showing SAT does not have poly-size circuits

is one route to proving P ≠ NP)• suppose SAT has poly-size circuits

– any consequences?– might hope: SAT P/poly PH collapses to P,

same as if SAT P

10/1//2013

Karp-Lipton

Theorem (KL): if SAT has poly-size circuits then PH collapses to the second level.

• Proof: – suffices to show Π2 Σ2

– L Π2 implies L expressible as:

L = { x : 8y 9z (x, y, z) R}

with R P.

10/1//2013

Karp-Lipton

L = { x : 8y 9z (x, y, z) R}

– given (x, y), “9z (x, y, z) R?” is in NP– pretend C solves SAT, use self-reducibility– Claim: if SAT P/poly, then L =

{ x : 9C 8y

[use C repeatedly to find some z for which (x, y, z) R; accept iff

(x, y, z) R] }

poly time

10/1//2013

Karp-Lipton

L = { x : 8y 9z (x, y, z) R}

{x : 9C 8y [use C repeatedly to find some z for which (x,y,z) R; accept iff (x,y,z) R] }

– x L: • some C decides SAT 9C 8y […] accepts

– x L: • 9y 8z (x, y, z) R 9C 8y […] rejects

10/1//2013 42

Randomized complexity classes

• model: probabilistic Turing Machine– deterministic TM with additional read-only

tape containing “coin flips”• BPP (Bounded-error Probabilistic Poly-time)

– L BPP if there is a p.p.t. TM M:

x L Pry[M(x,y) accepts] ≥ 2/3

x L Pry[M(x,y) rejects] ≥ 2/3

– “p.p.t” = probabilistic polynomial time

10/1//2013

BPP PH

• Recall: don’t know BPP different from EXP

Theorem (S,L,GZ): BPP (Π2Σ2)

• don’t know Π2Σ2 different from EXP but believe much weaker

10/1//2013

BPP PH

• Proof: – BPP language L: p.p.t. TM M:



– strong error reduction: p.p.t. TM M’• use n random bits (|y’| = n)• # strings y’ for which M’(x, y’) incorrect is at

most 2n/3

• (can’t achieve with naïve amplification)

10/1//2013

BPP PH

• view y’ = (w, z), each of length n/2• consider output of M’(x, (w, z)):

111 1

11

1 111 1 1

111 1

110 1

11

0 100 1 1

011 1

111 1

11

1 100 0 1

10 0

0 101 1

01

1 111 1 1

01 1

0

000 0

01

0 010 0 0

00 0

0001 0

00

0 000 0 0

01 00 00

0 000

0 000 0 0

00 0

0 000 0

01

0 010 0 0

00 0

0

w = 000…00 000…01 000…10 … 111…11

…

…

xL

xL

so many ones,

some disk is all ones

so few ones, not

enough for whole disk

10/1//2013

BPP PH

• proof (continued):– strong error reduction: # bad y’ < 2n/3 – y’ = (w, z) with |w| = |z| = n/2– Claim: L = {x : 9w 8z M’(x, (w, z)) = 1 }– xL: suppose 8w9z M’(x, (w, z)) = 0

• implies 2n/2 0’s; contradiction

– xL: suppose 9w8z M’(x, (w, z)) = 1• implies 2n/2 1’s; contradiction

10/1//2013

BPP PH

– given BPP language L: p.p.t. TM M:



– showed L = {x : 9w 8z M’(x, (w, z)) = 1}

– thus BPP Σ2

– BPP closed under complement BPP Π2

– conclude: BPP (Π2Σ2)

10/1//2013

New Topic

The complexity of

counting

10/1//2013

Counting problems

• So far, we have ignored function problems – given x, compute f(x)

• important class of function problems:

counting problems

– e.g. given 3-CNF φ how many satisfying assignments are there?

10/1//2013

Counting problems

• #P is the class of function problems expressible as:

input x f(x) = |{y : (x, y) R}|

where R P.

• compare to NP (decision problem)

input x f(x) = y : (x, y) R ?

where R P.

10/1//2013

Counting problems

• examples– #SAT: given 3-CNF φ how many satisfying

assignments are there?

– #CLIQUE: given (G, k) how many cliques of size at least k are there?

10/1//2013

Reductions

• Reduction from function problem f1 to function problem f2

– two efficiently computable functions Q, A

x(prob.

1)

y(prob.

2)

f2(y)f1(x)

Q

A

f2f1

10/1//2013

Reductions• problem f is #P-complete if

– f is in #P– every problem in #P

reduces to f

• “parsimonious reduction”: A is identity – many standard NP-completeness reductions

are parsimonious– therefore: if #SAT is #P-complete we get lots

of #P-complete problems

x(prob.

1)

y(prob.

2)

f2(y)f1(x)

Q

Af2f1

10/1//2013

#SAT

#SAT: given 3-CNF φ how many satisfying assignments are there?

Theorem: #SAT is #P-complete.

• Proof:– clearly in #P: (φ, A) R A satisfies φ– take any f #P defined by R P

10/1//2013

#SAT

– add new variables z, produce φ such that z φ(x, y, z) = 1 C(x, y) = 1

– for (x, y) such that C(x, y) = 1 this z is unique– hardwire x– # satisfying assignments = |{y : (x, y) R}|

…x… …y…

C


1 iff (x, y) R

f(x) =

|{y : (x, y) R}|

10/1//2013

Relationship to other classes

• To compare to classes of decision problems, usually consider

P#P

which is a decision class…• easy: NP, coNP P#P

• easy: P#P PSPACE

Toda’s Theorem : PH P#P.

fall 2013 cmu cs 15-855 computational complexity lecture 7 alternating quantifiers, ph, pspace....

Documents