the strange link between incompressibility and complexity
DESCRIPTION
The Strange Link between Incompressibility and Complexity. China Theory Week, Aarhus August 13, 2012. Today’s Goal:. To present new developments in a line of research dating back to 2002, presenting some unexpected connections between Kolmogorov Complexity (the theory of randomness), and - PowerPoint PPT PresentationTRANSCRIPT
Eric AllenderRutgers University
The Strange Link between Incompressibility and
Complexity
The Strange Link between Incompressibility and
Complexity
China Theory Week, Aarhus
August 13, 2012
Eric Allender: The Strange Link between Incompressibility and Complexity < 2 >
Today’s Goal:Today’s Goal:
To present new developments in a line of research dating back to 2002, presenting some unexpected connections between
– Kolmogorov Complexity (the theory of randomness), and
– Computational Complexity Theory
Which ought to have nothing to do with each other!
Eric Allender: The Strange Link between Incompressibility and Complexity < 3 >
Complexity ClassesComplexity Classes
P
NP
BPP
PSPACE
NEXP
EXPSPACE
P/poly
Eric Allender: The Strange Link between Incompressibility and Complexity < 4 >
A Jewel of DerandomizationA Jewel of Derandomization
[Impagliazzo, Wigderson, 1997]: If there is a problem computable in time 2n that requires circuits of size 2εn, then P = BPP.
Eric Allender: The Strange Link between Incompressibility and Complexity < 5 >
RandomnessRandomness
Which of these sequences did I obtain by flipping a coin?
0101010101010101010101010101010 0110100111111100111000100101100 1101010001011100111111011001010 1/π =0.3183098861837906715377675267450
Each of these sequences is equally likely, in the sense of
probability theory – which does not provide us with a way to
talk about “randomness”.
Eric Allender: The Strange Link between Incompressibility and Complexity < 6 >
RandomnessRandomness
Which of these sequences did I obtain by flipping a coin?
0101010101010101010101010101010 0110100111111100111000100101100 1101010001011100111111011001010 1/π =0.3183098861837906715377675267450
How many bits of information does each of these sequences
contain?
Eric Allender: The Strange Link between Incompressibility and Complexity < 7 >
Kolmogorov ComplexityKolmogorov Complexity
C(x) = min{|d| : U(d) = x}
– U is a “universal” Turing machine K(x) = min{|d| : U(d) = x}
– U is a “universal” prefix-free Turing machine Important property
– Invariance: The choice of the universal Turing machine U is unimportant (up to an additive constant).
x is random if C(x) ≥ |x|.
Eric Allender: The Strange Link between Incompressibility and Complexity < 8 >
Kolmogorov ComplexityKolmogorov Complexity
C(x) = min{|d| : U(d) = x}
– U is a “universal” Turing machine K(x) = min{|d| : U(d) = x}
– U is a “universal” prefix-free Turing machine Important property
– Invariance: The choice of the universal Turing machine U is unimportant (up to an additive constant).
x is random if C(x) ≥ |x|, or K(x) ≥ |x|.
Eric Allender: The Strange Link between Incompressibility and Complexity < 9 >
K, C, and RandomnessK, C, and Randomness
K(x) and C(x) are “close”:
– C(x) ≤ K(x) ≤ C(x) + 2 log |x| Two notions of randomness:
– RC = {x : C(x) ≥ |x|}
– RK = {x : K(x) ≥ |x|} …actually, infinitely many notions of
randomness:
– RCU = {x : CU(x) ≥ |x|}
Eric Allender: The Strange Link between Incompressibility and Complexity < 10 >
K, C, and RandomnessK, C, and Randomness
K(x) and C(x) are “close”:
– C(x) ≤ K(x) ≤ C(x) + 2 log |x| Two notions of randomness:
– RC = {x : C(x) ≥ |x|}
– RK = {x : K(x) ≥ |x|} …actually, infinitely many notions of
randomness:
– RCU = {x : CU(x) ≥ |x|}, RKU
= {x : KU(x) ≥ |x|}
Eric Allender: The Strange Link between Incompressibility and Complexity < 11 >
K, C, and RandomnessK, C, and Randomness
When it makes no difference, we’ll write “R” instead of RC or RK.
Basic facts:
– R is undecidable
– …but it is not “easy” to use it as an oracle.
– R is not NP-hard under poly-time ≤m reductions, unless P=NP.
– Things get more interesting when we consider more powerful types of reducibility.
Eric Allender: The Strange Link between Incompressibility and Complexity < 12 >
Three Bizarre InclusionsThree Bizarre Inclusions
NEXP is contained in NPR. [ABK06] PSPACE is contained in PR. [ABKMR06] BPP is contained in {A : A is poly-time ≤tt R}.
[BFKL10]
– A ≤tt reduction is a “non-adaptive” reduction.
– On input x, a list of queries is formulated before receiving any answer from the oracle.
Eric Allender: The Strange Link between Incompressibility and Complexity < 13 >
Three Bizarre InclusionsThree Bizarre Inclusions
NEXP is contained in NPR. [ABK06] PSPACE is contained in PR. [ABKMR06] BPP is contained in Ptt
R. [BFKL10]
“Bizarre”, because a non-computable “upper bound” is presented on complexity classes!
We have been unable to squeeze larger complexity classes inside. Are these containments optimal?
Eric Allender: The Strange Link between Incompressibility and Complexity < 14 >
Three Bizarre InclusionsThree Bizarre Inclusions
NEXP is contained in NPR. [ABK06] PSPACE is contained in PR. [ABKMR06] BPP is contained in Ptt
R. [BFKL10]
“Bizarre”, because a non-computable “upper bound” is presented on complexity classes!
If we restrict attention to RK, then we can do better…
Eric Allender: The Strange Link between Incompressibility and Complexity < 15 >
Three Bizarre InclusionsThree Bizarre Inclusions
NEXP is contained in NPRK.
– The decidable sets that are in NPRK for every U are in EXPSPACE. [AFG11]
PSPACE is contained in PRK.
BPP is contained in PttRK.
– The decidable sets that are in PttRK for every
U are in PSPACE. [AFG11]
Eric Allender: The Strange Link between Incompressibility and Complexity < 16 >
Three Bizarre InclusionsThree Bizarre Inclusions
NEXP is contained in NPRK (for every U).
– The decidable sets that are in NPRK for every U are in EXPSPACE. [AFG11]
PSPACE is contained in PRK (for every U).
BPP is contained in PttRK (for every U).
– The decidable sets that are in PttRK for
every U are in PSPACE. [AFG11]
– [CELM] The sets that are in PttRK for every U
are decidable.
Eric Allender: The Strange Link between Incompressibility and Complexity < 17 >
Three Bizarre InclusionsThree Bizarre Inclusions
NEXP is contained in NPRK (for every U).
– The decidable sets that are in NPRK for every U are in EXPSPACE. [AFG11]
PSPACE is contained in PRK (for every U).
BPP is contained in PttRK (for every U).
– The sets that are in PttRK for every U are in
PSPACE. [AFG11]
Eric Allender: The Strange Link between Incompressibility and Complexity < 18 >
Three Bizarre InclusionsThree Bizarre Inclusions
NEXP is contained in NPRK (for every U).
– The sets that are in NPRK for every U are in EXPSPACE. [AFG11]
PSPACE is contained in PRK (for every U).
BPP is contained in PttRK (for every U).
– The sets that are in PttRK for every U are in
PSPACE. [AFG11]
Eric Allender: The Strange Link between Incompressibility and Complexity < 19 >
Three Bizarre InclusionsThree Bizarre Inclusions
NEXP is contained in NPRK (for every U).
– The sets that are in NPRK for every U are in EXPSPACE. [AFG11]
Conjecture: This should hold for RC, too.
BPP is contained in PttRK (for every U).
– The sets that are in PttRK for every U are in
PSPACE. [AFG11]
Eric Allender: The Strange Link between Incompressibility and Complexity < 20 >
Three Bizarre InclusionsThree Bizarre Inclusions
NEXP is contained in NPRK (for every U).
– The sets that are in NPRK for every U are in EXPSPACE. [AFG11]
This holds even for sets in EXPttRK for all U!
BPP is contained in PttRK (for every U).
– The sets that are in PttRK for every U are in
PSPACE. [AFG11]
Eric Allender: The Strange Link between Incompressibility and Complexity < 21 >
Three Bizarre InclusionsThree Bizarre Inclusions
NEXP is contained in NPRK (for every U).
– The sets that are in NPRK for every U are in EXPSPACE. [AFG11]
Conjecture: This class is exactly NEXP.
BPP is contained in PttRK (for every U).
– The sets that are in PttRK for every U are in
PSPACE. [AFG11]
Conjecture: This class is exactly BPP.
Eric Allender: The Strange Link between Incompressibility and Complexity < 22 >
Three Bizarre InclusionsThree Bizarre Inclusions
NEXP is contained in NPRK (for every U).
– The sets that are in NPRK for every U are in EXPSPACE. [AFG11]
Conjecture: This class is exactly NEXP.
BPP is contained in PttRK (for every U).
– The sets that are in PttRK for every U are in
PSPACE. [AFG11]
Conjecture: This class is exactly BPP P.
Eric Allender: The Strange Link between Incompressibility and Complexity < 23 >
K-Complexity and BPP vs PK-Complexity and BPP vs P
BPP is contained in PttRK (for every U).
– The sets that are in PttRK for every U are in
PSPACE.
Conjecture: This class is exactly P. Some support for this conjecture [ABK06]:– The decidable sets that are in Pdtt
RK for every U are in P.
– The decidable sets that are in Pparity-ttRK for
every U are in P.
Eric Allender: The Strange Link between Incompressibility and Complexity < 24 >
K-Complexity and BPP vs PK-Complexity and BPP vs P
BPP is contained in PttRK (for every U).
– The sets that are in PttRK for every U are in
PSPACE.
Conjecture: This class is exactly P. New results support a weaker conjecture: Conjecture: This class is contained in
PSPACE ∩ P/poly. More strongly: Every decidable set in Ptt
R is in P/poly.
Eric Allender: The Strange Link between Incompressibility and Complexity < 25 >
K-Complexity and BPP vs PK-Complexity and BPP vs P
BPP is contained in PttRK (for every U).
– The sets that are in PttRK for every U are in
PSPACE.
Conjecture: This class is exactly P. New results support a weaker conjecture : Conjecture: This class is contained in
PSPACE ∩ P/poly. More strongly: Every decidable set in Ptt
R is in P/poly (i.e., for every U, and for both C and K).
Eric Allender: The Strange Link between Incompressibility and Complexity < 26 >
The Central ConjectureThe Central Conjecture
Conjecture: Every decidable set in PttR is in
P/poly. What can we show?
Eric Allender: The Strange Link between Incompressibility and Complexity < 27 >
The Central ConjectureThe Central Conjecture
Conjecture: Every decidable set in PttR is in
P/poly. What can we show? We show that a similar statement holds in the
context of time-bounded K-complexity.
Eric Allender: The Strange Link between Incompressibility and Complexity < 28 >
Time-Bounded K-complexityTime-Bounded K-complexity
Let t be a time bound. (Think of t as being large, such as Ackermann’s function.)
Define Kt(x) to be min{|d| : U(d) = x in at most t(|x|) steps}.
Define RKt to be {x : Kt(x) ≥ |x|}. Define TTRT = {A : A is in Ptt
RKt for all large enough time bounds t}.
Vague intuition: Poly-time reductions should not be able to distinguish between RKt and RK, for large t.
Eric Allender: The Strange Link between Incompressibility and Complexity < 29 >
The Central ConjectureThe Central Conjecture
Conjecture: Every decidable set in PttR is in
P/poly. We show that a similar statement holds in the
context of time-bounded K-complexity:
– TTRT is contained in P/poly [ABFL12]. If t(n) = 22n, then RKt is NOT in P/poly. …which supports our “vague intuition”,
because this set is not reducible to the time-t’-random strings for t’ >> t.
Eric Allender: The Strange Link between Incompressibility and Complexity < 30 >
The Central ConjectureThe Central Conjecture
Conjecture: Every decidable set in PttR is in
P/poly. We show that a similar statement holds in the
context of time-bounded K-complexity:
– TTRT is contained in P/poly [ABFL12]. BUT – The same P/poly bound holds, even if
we consider PRKt instead of PttRKt.
…and recall PSPACE is contained in PR. So the “vague intuition” is wrong!
Eric Allender: The Strange Link between Incompressibility and Complexity < 31 >
The Central Conjecture: An Earlier Approach
The Central Conjecture: An Earlier Approach
Conjecture: Every decidable set in PttR is in
P/poly. We give a proof of a statement of the form:
An
AjΨ(n,j)
such that: if for each n and j
there is a proof in PA of Ψ(n,j)
then the conjecture holds.
Eric Allender: The Strange Link between Incompressibility and Complexity < 32 >
Basic Proof TheoryBasic Proof Theory
Recall that Peano Arithmetic cannot prove the statement “PA is consistent”.
Let PA1 be PA + “PA is consistent”. Similarly, one can define PA2, PA3, … “PA is consistent” can be formulated as “for all
j, there is no length j proof of 0=1”. For each j, PA can prove “there is no length j
proof of 0=1”.
Eric Allender: The Strange Link between Incompressibility and Complexity < 33 >
The Central Conjecture: An Earlier Approach
The Central Conjecture: An Earlier Approach
Conjecture: Every decidable set in PttR is in
P/poly. We give a proof (in PA1) of a statement of the
form: An
AjΨ(n,j)
such that: if for each n and j
there is a proof in PA of Ψ(n,j)
then the conjecture holds.
Eric Allender: The Strange Link between Incompressibility and Complexity < 34 >
The Central Conjecture: The Earlier Approach FailsThe Central Conjecture:
The Earlier Approach Fails The connections to proof theory were
unexpected and intriguing, and seemed promising…
But unfortunately, it turns out that many of the statements Ψ(n,j) are independent of PA (and a related approach yields statements Ψ(n,j,k) that are independent of each system PAr).
Eric Allender: The Strange Link between Incompressibility and Complexity < 35 >
A High-Level View of the “Earlier Approach”
A High-Level View of the “Earlier Approach”
Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).
Then there is a d such that for all x,
there is a V containing only strings of length at most d+log f(|x|), such that
MV(x) = A(x).
Note: V says “long queries are non-random”.
Eric Allender: The Strange Link between Incompressibility and Complexity < 36 >
A Warm-UpA Warm-Up
Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).
Then there is a d such that for all x,
there is a V containing only strings of length at most d+log f(|x|), such that
MV(x) = A(x).
Note: If some V works for all x of length n, then A is in P/poly.
Eric Allender: The Strange Link between Incompressibility and Complexity < 37 >
ProofProof
Assume that for each d there is some x such that, for all V containing strings of length at most d+log f(|x|), MV(x)≠A(x).
Consider the machine that takes input (d,r) and finds x (as above) and outputs the rth element of Q(x).
This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1)
Eric Allender: The Strange Link between Incompressibility and Complexity < 38 >
ProofProof
Assume that for each d there is some x such that, for all V containing strings of length at most d+log f(|x|), MV(x)≠A(x).
Consider the machine that takes input (d,r) and finds x (as above) and outputs the rth element of Q(x).
This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1) < d + log f(|x|).
Thus if we pick V* to be R∩{0,1}d+log f(|x|), we see that MV*(x) = MR(x)
Eric Allender: The Strange Link between Incompressibility and Complexity < 39 >
ProofProof
Assume that for each d there is some x such that, for all V containing strings of length at most d+log f(|x|), MV(x)≠A(x).
Consider the machine that takes input (d,r) and finds x (as above) and outputs the rth element of Q(x).
This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1) < d + log f(|x|).
Thus if we pick V* to be R∩{0,1}d+log f(|x|), we see that MV*(x) = MR(x) = A(x). Contradiction!
Eric Allender: The Strange Link between Incompressibility and Complexity < 40 >
Cleaning Things UpCleaning Things Up
Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).
Then there is a d such that for all x,
there is a V containing only strings of length at most d+log f(|x|) gA(|x|), such that
MV(x) = A(x).
Eric Allender: The Strange Link between Incompressibility and Complexity < 41 >
Cleaning Things UpCleaning Things Up
Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).
Then there is a d such that for all x,
there is a V containing only strings of length at most d+log f(|x|) gA(|x|), such that
MV(x) = A(x).
Eric Allender: The Strange Link between Incompressibility and Complexity < 42 >
Cleaning Things UpCleaning Things Up
Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).
Then for all x,
there is a V containing only strings in R of length at most gA(|x|) such that
MV(x) = A(x).
Eric Allender: The Strange Link between Incompressibility and Complexity < 43 >
A RefinementA Refinement
Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).
Then for all x,
there is a V containing only strings in R of length at most gA(|x|) such that
MV(x) = A(x).
Eric Allender: The Strange Link between Incompressibility and Complexity < 44 >
Approximating RApproximating R
We can obtain a series of approximations to R (up to length gA(n)) as follows:
Rn,0 = all strings of length at most gA(n). Rn,i+1 = Rn,i minus the i+1st string of length at
most gA(n) that is found, in an enumeration of non-random strings.
Rn,0, Rn,1, Rn,2, … Rn,i* = R∩{0,1}gA(n)
Eric Allender: The Strange Link between Incompressibility and Complexity < 45 >
A RefinementA Refinement
Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).
Then for all xє{0,1}n, for all i,
there is a V containing only strings in Rn,i such that
MV(x) = A(x).
Eric Allender: The Strange Link between Incompressibility and Complexity < 46 >
ProofProof
Assume that for each d there is some x,i such that, for all V containing strings in Rn,i of length at most d+log f(|x|), MV(x)≠A(x).
Consider the machine that takes input (d,r) and finds x,i (as above) and outputs the rth element of Q(x).
This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1) < d + log f(|x|).
Thus if we pick V* to be R∩{0,1}d+log f(|x|), we see that MV*(x) = MR(x) = A(x). Contradiction!
Eric Allender: The Strange Link between Incompressibility and Complexity < 47 >
Where does PA come in??Where does PA come in??
Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).
Then for all xє{0,1}n, for all i,
there is a V containing only strings in Rn,i such that
MV(x) = A(x)
And for all k, there is not a length-k proof that “for all i, V is not equal to Rn,i”.
Eric Allender: The Strange Link between Incompressibility and Complexity < 48 >
What went wrong with the earlier approach.
What went wrong with the earlier approach.
We have shown: For all xє{0,1}n, for all i, there is a V containing only short strings in Rn,i such that MV(x) = A(x).
We were aiming at showing that one can swap the quantifiers, so that for all n, there is a V containing only short strings in Rn,i such that, for all x of length n, MV(x) = A(x).
But there is a (useless) reduction M for which this is false. (M already knows the outcome of its queries, assuming that the oracle is R.)
Eric Allender: The Strange Link between Incompressibility and Complexity < 49 >
Open Questions:Open Questions:
Decrease the gap (NEXP vs EXPSPACE) between the lower and upper bounds on the complexity of the problems that are in NPRK for every U.
Some of our proofs rely on using RK. Do similar results hold also for RC?
– Disprove: The halting problem is in PttRC.
Can the PSPACE ∩ P/poly bound (in the time-bounded setting) be improved to BPP?
Is this approach relevant at all to the P=BPP question?
Eric Allender: The Strange Link between Incompressibility and Complexity < 50 >
P vs BPPP vs BPP
Our main intuition for P=BPP comes from [Impagliazzo, Wigderson]. Circuit lower bounds imply derandomization.
Note that this provides much more than “merely” P=BPP; it gives a recipe for simulating any probabilistic algorithm.
Goldreich has argued that any proof of P=BPP actually yields pseudorandom generators (and hence a “recipe” as above)…
– …but this has only been proved for the “promise problem” formulation of P=BPP.
Eric Allender: The Strange Link between Incompressibility and Complexity < 51 >
P vs BPPP vs BPP
Recall that TTRT sits between BPP and PSPACE ∩ P/poly.
A proof that TTRT = P would show that BPP = P – but it is not clear that this would yield any sort of recipe for constructing useful pseudorandom generators.
Although it would be a less “useful” approach, perhaps it might be an easier approach?
Eric Allender: The Strange Link between Incompressibility and Complexity < 52 >
How to use R as an OracleHow to use R as an Oracle
Eric Allender: The Strange Link between Incompressibility and Complexity < 53 >
Circuit ComplexityCircuit Complexity
Let D be a circuit of AND and OR gates (with negations at the inputs). Size(D) = # of wires in D.
Size(f) = min{Size(D) : D computes f} We may allow oracle gates for a set A, along
with AND and OR gates. SizeA(f) = min{Size(D) : DA computes f}
Eric Allender: The Strange Link between Incompressibility and Complexity < 54 >
Oracle GatesOracle Gates
An “oracle gate” for B in a circuit:
B
Eric Allender: The Strange Link between Incompressibility and Complexity < 55 >
K-complexity ≈ Circuit ComplexityK-complexity ≈ Circuit Complexity
There are some obvious similarities in the definitions.
C(x) = min{|d| : U(d) = x} Size(f) = min{Size(D) : D computes f}
Eric Allender: The Strange Link between Incompressibility and Complexity < 56 >
K-complexity ≈ Circuit ComplexityK-complexity ≈ Circuit Complexity
There are some obvious similarities in the definitions. What are some differences?
A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings.
– Given any string x, let fx be the function whose truth table is the string of length 2log|x|, padded out with 0’s, and define Size(x) to be Size(fx).
Eric Allender: The Strange Link between Incompressibility and Complexity < 57 >
K-complexity ≈ Circuit ComplexityK-complexity ≈ Circuit Complexity
There are some obvious similarities in the definitions. What are some differences?
A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings.
A more fundamental difference:
– C(x) is not computable; Size(x) is.
Eric Allender: The Strange Link between Incompressibility and Complexity < 58 >
Lightning Review: ComputabilityLightning Review: Computability
Here’s all we’ll need to know about computability:– The halting problem, everyone’s favorite
uncomputable problem:– H = {(i,x) : Mi halts on input x}– Every r.e. problem is poly-time many-one
reducible to H.
– One such r.e. problem: {x : C(x) < |x|}. – There is no time-bounded many-one
reduction in other direction, but there is a (large) time t Turing reduction (and hence C is not computable).
Eric Allender: The Strange Link between Incompressibility and Complexity < 59 >
K-complexity ≈ Circuit ComplexityK-complexity ≈ Circuit Complexity
There are some obvious similarities in the definitions. What are some differences?
A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings.
A more fundamental difference:
– C(x) is not computable; Size(x) is. In fact, there is a fascinating history, regarding
the complexity of the Size function.
Eric Allender: The Strange Link between Incompressibility and Complexity < 60 >
MCSPMCSP
MCSP = {(x,i) : Size(x) < i}. MCSP is in NP, but is not known to be NP-
complete. Some history: NP-completeness was
discovered independently in the 70s by Cook (in North America) and Levin (in Russia).
Levin delayed publishing his results, because he wanted to show that MCSP was NP-complete, thinking that this was the more important problem.
Eric Allender: The Strange Link between Incompressibility and Complexity < 61 >
MCSPMCSP
MCSP = {(x,i) : Size(x) < i}. Why was Levin so interested in MCSP? In the USSR in the 70’s, there was great
interest in problems requiring “perebor”, or “brute-force search”. For various reasons, MCSP was a focal point of this interest.
It was recognized at the time that this was “similar in flavor” to questions about resource-bounded Kolmogorov complexity, but there were no theorems along this line.
Eric Allender: The Strange Link between Incompressibility and Complexity < 62 >
MCSPMCSP
MCSP = {(x,i) : Size(x) < i}. 3 decades later, what do we know about
MCSP? If it’s complete under the “usual” poly-time
many-one reductions, then BPP = P. MCSP is not believed to be in P. We know:
– Factoring is in BPPMCSP.
– Every cryptographically-secure one-way function can be inverted in PMCSP/poly.
Eric Allender: The Strange Link between Incompressibility and Complexity < 63 >
So how can K-complexity and Circuit complexity be the same?So how can K-complexity and
Circuit complexity be the same? C(x) ≈ SizeH(x), where H is the halting
problem. For one direction, let U(d) = x. We need a
small circuit (with oracle gates for H) for fx, where fx(i) is the i-th bit of x. This is easy, since {(d,i,b) : U(d) outputs a string whose i-th bit is b} is computably-enumerable.
For the other direction, let SizeH(fx) = m. No oracle gate has more than m wires coming into it. Given a description of D (size not much bigger than m) and the m-bit number giving the size of {y in H : |y| ≤ m}, U can simulate DH and produce fx.
Eric Allender: The Strange Link between Incompressibility and Complexity < 64 >
So how can K-complexity and Circuit complexity be the same?So how can K-complexity and
Circuit complexity be the same? C(x) ≈ SizeH(x), where H is the halting
problem. So there is a connection between C(x) and
Size(x) … …but is it useful? First, let’s look at decidable versions of
Kolmogorov complexity.
Eric Allender: The Strange Link between Incompressibility and Complexity < 65 >
Time-Bounded Kolmogorov Complexity
Time-Bounded Kolmogorov Complexity
The usual definition: Ct(x) = min{|d| : U(d) = x in time t(|x|)}. Problems with this definition
– No invariance! If U and U’ are different universal Turing machines, Ct
U and CtU’ have
no clear relationship.
– (One can bound CtU by Ct’
U’ for t’ slightly larger than t – but nothing can be done for t’=t.)
No nice connection to circuit complexity!
Eric Allender: The Strange Link between Incompressibility and Complexity < 66 >
Time-Bounded Kolmogorov Complexity
Time-Bounded Kolmogorov Complexity
Levin’s definition: Kt(x) = min{|d|+log t : U(d) = x in time t}. Invariance holds! If U and U’ are different
universal Turing machines, KtU(x) and KtU’(x) are within log |x| of each other.
And, there’s a connection to Circuit Complexity:
– Let A be complete for E = Dtime(2O(n)). Then Kt(x) ≈ SizeA(x).
Eric Allender: The Strange Link between Incompressibility and Complexity < 67 >
Time-Bounded Kolmogorov Complexity
Time-Bounded Kolmogorov Complexity
Levin’s definition: Kt(x|φ) = min{|d|+log t : U(d,φ)=x in time t}. Why log t?
– This gives an optimal search order for NP search problems.
– This algorithm finds satisfying assignments in poly time, if any algorithm does:
– On input φ, search through assignments y in order of increasing Kt(y|φ).
Eric Allender: The Strange Link between Incompressibility and Complexity < 68 >
Time-Bounded Kolmogorov Complexity
Time-Bounded Kolmogorov Complexity
Levin’s definition: Kt(x) = min{|d|+log t : U(d) = x in time t}. Why log t?
– This gives an optimal search order for NP search problems.
– Adding t instead of log t would give every string complexity ≥ |x|.
…So let’s look for a sensible way to allow the run-time to be much smaller.
Eric Allender: The Strange Link between Incompressibility and Complexity < 69 >
Revised Kolmogorov ComplexityRevised Kolmogorov Complexity
C(x) = min{|d| : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x} (where bit # i+1 of x is *).
– This is identical to the original definition. Kt(x) = min{|d|+log t : for all i ≤ |x| + 1, U(d,i,b)
= 1 iff b is the i-th bit of x, in time t}.
– The new and old definitions are within O(log |x|) of each other.
Define KT(x) = min{|d|+t : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t}.
Eric Allender: The Strange Link between Incompressibility and Complexity < 70 >
Kolmogorov Complexity is Circuit Complexity
Kolmogorov Complexity is Circuit Complexity
C(x) ≈ SizeH(x). Kt(x) ≈ SizeE(x). KT(x) ≈ Size(x). Other measures of complexity can be
captured in this way, too:
– Branching Program Size ≈ KB(x) = min{|d|+2s : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in space s}.
Eric Allender: The Strange Link between Incompressibility and Complexity < 71 >
Kolmogorov Complexity is Circuit Complexity
Kolmogorov Complexity is Circuit Complexity
C(x) ≈ SizeH(x). Kt(x) ≈ SizeE(x). KT(x) ≈ Size(x). Other measures of complexity can be
captured in this way, too:
– Formula Size ≈ KF(x) = min{|d|+2t : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t}, for an alternating Turing machine U.
Eric Allender: The Strange Link between Incompressibility and Complexity < 72 >
…but is this interesting?…but is this interesting?
The result that Factoring is in BPPMCSP was first proved by observing that, in PMCSP, one can accept a large set of strings having large KT complexity: RKT = {x : KT(x) ≥ |x|}.
(Basic Idea): There is a pseudorandom generator based on factoring, such that factoring is in BPPT for any test T that distinguishes truly random strings from pseudorandom strings. RKT is just such a test T.
Eric Allender: The Strange Link between Incompressibility and Complexity < 73 >
This idea has many variants.This idea has many variants.
Consider RKT, RKt, and RC. RKT is in coNP, and not known to be coNP hard. RC is not hard for NP under poly-time many-one
reductions, unless P=NP.
– How about more powerful reductions?
– Is there anything interesting that we could compute quickly if C were computable, that we can’t already compute quickly?
– Proof uses PRGs, Interactive Proofs, and the fact that an element of RC of length n can be found in
But RC is undecidable! Perhaps H is in P relative to RC?
Eric Allender: The Strange Link between Incompressibility and Complexity < 74 >
This idea has many variants.This idea has many variants.
Consider RKT, RKt, and RC. RKT is in coNP, and not known to be coNP hard. RC is not hard for NP under poly-time many-one
reductions, unless P=NP.
– How about more powerful reductions? We show:
– PSPACE is in P relative to RC.
– NEXP is in NP relative to RC.
– Proof uses PRGs, Interactive Proofs, and the fact that an element of RC of length n can be found in poly time, relative to RC [BFNV].
But RC is undecidable! Perhaps H is in P relative to RC?
Eric Allender: The Strange Link between Incompressibility and Complexity < 75 >
Relationship between H and RCRelationship between H and RC
Perhaps H is in P relative to RC? This is still open. It is known that there is a
computable time bound t such that H is in DTime(t) relative to RC [Kummer].
…but the bound t depends on the choice of U in the definition of C(x).
We also showed: H is in P/poly relative to RC. Thus, in some sense, there is an “efficient”
reduction of H to RC.
Eric Allender: The Strange Link between Incompressibility and Complexity < 76 >
Playing the Game in PSPACEPlaying the Game in PSPACE
We can also define a space-bounded measure KS, such that Kt(x) < KS(x) < KT(x).
RKS is complete for PSPACE under BPP reductions (and, in fact, even under “ZPP” reductions, which implies completeness under NP reductions and P/poly reductions).
[This makes use of an even stronger form of the Impagliazzo-Wigderson generator, which relies on the “hard” function being “downward self-reducible and random self-reducible”.]