Download - Curiouser and Curiouser : The Link between Incompressibility and Complexity

Eric AllenderRutgers University

Curiouser and Curiouser: The Link between

Incompressibility and Complexity

CiE Special Session, June 19, 2012

Eric Allender: Curiouser & Curiouser: The Link between Incompressibility and Complexity < 2 >

Today’s Goal:

To present new developments in a line of research dating back to 2002, presenting some unexpected connections between– Kolmogorov Complexity (the theory of

randomness), and– Computational Complexity Theory

Which ought to have nothing to do with each other!


Complexity Classes

P

NP

BPP

PSPACENEX

P

EXPSPACE

P/poly


A Jewel of Derandomization [Impagliazzo, Wigderson, 1997]: If there is a

problem computable in time 2n that requires circuits of size 2εn, then P = BPP.


Kolmogorov Complexity

C(x) = min{|d| : U(d) = x}– U is a “universal” Turing machine

K(x) = min{|d| : U(d) = x}– U is a “universal” prefix-free Turing machine

Important property– Invariance: The choice of the universal

Turing machine U is unimportant (up to an additive constant).

x is random if C(x) ≥ |x|.


Kolmogorov Complexity

C(x) = min{|d| : U(d) = x}– U is a “universal” Turing machine

K(x) = min{|d| : U(d) = x}– U is a “universal” prefix-free Turing machine

Important property– Invariance: The choice of the universal

Turing machine U is unimportant (up to an additive constant).

x is random if C(x) ≥ |x|, or K(x) ≥ |x|.


K, C, and Randomness K(x) and C(x) are “close”:– C(x) ≤ K(x) ≤ C(x) + 2 log |x|

Two notions of randomness:– RC = {x : C(x) ≥ |x|}

– RK = {x : K(x) ≥ |x|} …actually, infinitely many notions of

randomness:

– RCU = {x : CU(x) ≥ |x|}


K, C, and Randomness K(x) and C(x) are “close”:– C(x) ≤ K(x) ≤ C(x) + 2 log |x|

Two notions of randomness:– RC = {x : C(x) ≥ |x|}

– RK = {x : K(x) ≥ |x|} …actually, infinitely many notions of

randomness:

– RCU = {x : CU(x) ≥ |x|}, RKU

= {x : KU(x) ≥ |x|}


K, C, and Randomness When it makes no difference, we’ll write “R”

instead of RC or RK. Basic facts:– R is undecidable– …but it is not “easy” to use it as an oracle.– R is not NP-hard under poly-time ≤m

reductions, unless P=NP.– Things get more interesting when we

consider more powerful types of reducibility.


Three Bizarre Inclusions NEXP is contained in NPR. [ABK06] PSPACE is contained in PR. [ABKMR06] BPP is contained in {A : A is poly-time ≤tt R}.

[BFKL10]– A ≤tt reduction is a “non-adaptive” reduction.– On input x, a list of queries is formulated

before receiving any answer from the oracle.


Three Bizarre Inclusions NEXP is contained in NPR. [ABK06] PSPACE is contained in PR. [ABKMR06] BPP is contained in Ptt

R. [BFKL10]

“Bizarre”, because a non-computable “upper bound” is presented on complexity classes!

We have been unable to squeeze larger complexity classes inside. Are these containments optimal?


Three Bizarre Inclusions NEXP is contained in NPR. [ABK06] PSPACE is contained in PR. [ABKMR06] BPP is contained in Ptt

R. [BFKL10]

“Bizarre”, because a non-computable “upper bound” is presented on complexity classes!

If we restrict attention to RK, then we can do better…


Three Bizarre Inclusions NEXP is contained in NPRK.– The decidable sets that are in NPRK for every

U are in EXPSPACE. [AFG11] PSPACE is contained in PRK.

BPP is contained in PttRK.

– The decidable sets that are in PttRK for every

U are in PSPACE. [AFG11]


Three Bizarre Inclusions NEXP is contained in NPRK (for every U).– The decidable sets that are in NPRK for every

U are in EXPSPACE. [AFG11] PSPACE is contained in PRK (for every U).

BPP is contained in PttRK (for every U).

– The decidable sets that are in PttRK for

every U are in PSPACE. [AFG11]– [CELM] The sets that are in Ptt

RK for every U are decidable.


Three Bizarre Inclusions NEXP is contained in NPRK (for every U).– The decidable sets that are in NPRK for every

U are in EXPSPACE. [AFG11] PSPACE is contained in PRK (for every U).


– The sets that are in PttRK for every U are in

PSPACE. [AFG11]


Three Bizarre Inclusions NEXP is contained in NPRK (for every U).– The sets that are in NPRK for every U are in

EXPSPACE. [AFG11] PSPACE is contained in PRK (for every U).



PSPACE. [AFG11]



EXPSPACE. [AFG11] Conjecture: This should hold for RC, too.



PSPACE. [AFG11]



EXPSPACE. [AFG11] This holds even for sets in EXPtt

RK for all U!



PSPACE. [AFG11]



EXPSPACE. [AFG11] Conjecture: This class is exactly NEXP.



PSPACE. [AFG11]Conjecture: This class is exactly BPP.



EXPSPACE. [AFG11] Conjecture: This class is exactly NEXP.



PSPACE. [AFG11]Conjecture: This class is exactly BPP P.


K-Complexity and BPP vs P BPP is contained in Ptt

RK (for every U).– The sets that are in Ptt

RK for every U are in PSPACE.

Conjecture: This class is exactly P. Some support for this conjecture [ABK06]:– The decidable sets that are in Pdtt

RC for every U are in P.

– The decidable sets that are in Pparity-ttRC for

every U are in P.





Conjecture: This class is exactly P. New results support a weaker conjecture: Conjecture: This class is contained in

PSPACE ∩ P/poly. More strongly: Every decidable set in Ptt

R is in P/poly.





Conjecture: This class is exactly P. New results support a weaker conjecture : Conjecture: This class is contained in

PSPACE ∩ P/poly. More strongly: Every decidable set in Ptt

R is in P/poly (i.e., for every U, and for both C and K).


The Central Conjecture Conjecture: Every decidable set in Ptt

R is in P/poly.

What can we show?



R is in P/poly.

What can we show? We show that a similar statement holds in the

context of time-bounded K-complexity.


Time-Bounded K-complexity Let t be a time bound. (Think of t as being

large, such as Ackermann’s function.) Define Kt(x) to be min{|d| : U(d) = x in at most

t(|x|) steps}. Define RKt to be {x : Kt(x) ≥ |x|}. Define TTRT = {A : A is in Ptt

RKt for all large enough time bounds t}.

Vague intuition: Poly-time reductions should not be able to distinguish between RKt and RK, for large t.



R is in P/poly.

We show that a similar statement holds in the context of time-bounded K-complexity:– TTRT is contained in P/poly [ABFL12].

If t(n) = 22n, then RKt is NOT in P/poly. …which supports our “vague intuition”,

because this set is not reducible to the time-t’-random strings for t’ >> t.



R is in P/poly.

We show that a similar statement holds in the context of time-bounded K-complexity:– TTRT is contained in P/poly [ABFL12].

BUT – The same P/poly bound holds, even if we consider PRKt instead of Ptt

RKt. …and recall PSPACE is contained in PR. So the “vague intuition” is wrong!


The Central Conjecture: An Earlier Approach

Conjecture: Every decidable set in PttR is in

P/poly. We give a proof of a statement of the form:

An

AjΨ(n,j)

such that: if for each n and jthere is a proof in PA of Ψ(n,j)then the conjecture holds.


Basic Proof Theory Recall that Peano Arithmetic cannot prove the

statement “PA is consistent”. Let PA1 be PA + “PA is consistent”. Similarly, one can define PA2, PA3, … “PA is consistent” can be formulated as “for all

j, there is no length j proof of 0=1”. For each j, PA can prove “there is no length j

proof of 0=1”.


The Central Conjecture: An Earlier Approach

Conjecture: Every decidable set in PttR is in

P/poly. We give a proof (in PA1) of a statement of the

form: An

AjΨ(n,j)

such that: if for each n and jthere is a proof in PA of Ψ(n,j)then the conjecture holds.


The Central Conjecture: The Earlier Approach Fails

The connections to proof theory were unexpected and intriguing, and seemed promising…

But unfortunately, it turns out that many of the statements Ψ(n,j) are independent of PA (and a related approach yields statements Ψ(n,j,k) that are independent of each system PAr).


A High-Level View of the “Earlier Approach”

Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).

Then there is a d such that for all x,there is a V containing only strings of length at

most d+log f(|x|), such thatMV(x) = A(x).Note: V says “long queries are non-random”.


A Warm-Up Let A be decidable, and let M be a poly-time

machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).


most d+log f(|x|), such thatMV(x) = A(x).Note: If some V works for all x of length n, then A

is in P/poly.


Proof Assume that for each d there is some x such

that, for all V containing strings of length at most d+log f(|x|), MV(x)≠A(x).

Consider the machine that takes input (d,r) and finds x (as above) and outputs the rth element of Q(x).

This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1)





This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1) < d + log f(|x|).

Thus if we pick V* to be R∩{0,1}d+log f(|x|), we see that MV*(x) = MR(x)






Thus if we pick V* to be R∩{0,1}d+log f(|x|), we see that MV*(x) = MR(x) = A(x). Contradiction!


Cleaning Things Up Let A be decidable, and let M be a poly-time



most d+log f(|x|) gA(|x|), such thatMV(x) = A(x).





most d+log f(|x|) gA(|x|), such thatMV(x) = A(x).




Then for all x,there is a V containing only strings in R of length

at most gA(|x|) such thatMV(x) = A(x).


A Refinement Let A be decidable, and let M be a poly-time


Then for all x,there is a V containing only strings in R of length

at most gA(|x|) such thatMV(x) = A(x).


Approximating R We can obtain a series of approximations to R

(up to length gA(n)) as follows: Rn,0 = all strings of length at most gA(n). Rn,i+1 = Rn,i minus the i+1st string of length at

most gA(n) that is found, in an enumeration of non-random strings.

Rn,0, Rn,1, Rn,2, … Rn,i* = R∩{0,1}gA(n)


A Refinement Let A be decidable, and let M be a poly-time


Then for all xє{0,1}n, for all i,there is a V containing only strings in Rn,i such

thatMV(x) = A(x).


Proof Assume that for each d there is some x,i such

that, for all V containing strings in Rn,i of length at most d+log f(|x|), MV(x)≠A(x).

Consider the machine that takes input (d,r) and finds x,i (as above) and outputs the rth element of Q(x).


Thus if we pick V* to be R∩{0,1}d+log f(|x|), we see that MV*(x) = MR(x) = A(x). Contradiction!


Where does PA come in?? Let A be decidable, and let M be a poly-time


Then for all xє{0,1}n, for all i,there is a V containing only strings in Rn,i such

thatMV(x) = A(x)and there is not a length-k proof that “for all i, V

is not equal to Rn,i”.


What went wrong with the earlier approach.

We have shown: For all xє{0,1}n, for all i, there is a V containing only short strings in Rn,i such that MV(x) = A(x).

We were aiming at showing that one can swap the quantifiers, so that for all n, there is a V containing only short strings in Rn,i such that, for all x of length n, MV(x) = A(x).

But there is a (useless) reduction M for which this is false. (M already knows the outcome of its queries, assuming that the oracle is R.)


Open Questions: Decrease the gap (NEXP vs EXPSPACE)

between the lower and upper bounds on the complexity of the problems that are in NPRK for every U.

Some of our proofs rely on using RK. Do similar results hold also for RC?– Disprove: The halting problem is in Ptt

RC. Can the PSPACE ∩ P/poly bound (in the time-

bounded setting) be improved to BPP? Is this approach relevant at all to the P=BPP

question?


P vs BPP Our main intuition for P=BPP comes from

[Impagliazzo, Wigderson]. Circuit lower bounds imply derandomization.

Note that this provides much more than “merely” P=BPP; it gives a recipe for simulating any probabilistic algorithm.

Goldreich has argued that any proof of P=BPP actually yields pseudorandom generators (and hence a “recipe” as above)…– …but this has only been proved for the

“promise problem” formulation of P=BPP.


P vs BPP Recall that TTRT sits between BPP and

PSPACE ∩ P/poly. A proof that TTRT = P would show that BPP =

P – but it is not clear that this would yield any sort of recipe for constructing useful pseudorandom generators.

Although it would be a less “useful” approach, perhaps it might be an easier approach?


Thank you!

Download - Curiouser and Curiouser : The Link between Incompressibility and Complexity

Top Related