eric allender rutgers university circuit complexity, kolmogorov complexity, and prospects for lower...

Eric AllenderRutgers University

Circuit Complexity, Kolmogorov Complexity, and Prospects for

Lower Bounds

Circuit Complexity, Kolmogorov Complexity, and Prospects for

Lower Bounds

DCFS 2008

Eric Allender: How Close Are We to Proving Circuit Lower Bounds? < 2 >

Today’s Goal:Today’s Goal:

To raise awareness of the tight connection between circuit complexity and Kolmogorov complexity.

And to show that this is useful. To plant seeds of optimism, regarding the

prospects of proving lower bounds in circuit complexity.


Kolmogorov ComplexityKolmogorov Complexity

C(x) = min{|d| : U(d) = x} Important property

– Invariance: The choice of the universal Turing machine U is unimportant.

x is random if C(x) ≥ |x|. CA(x) = min{|d| : UA(d) = x}


Circuit ComplexityCircuit Complexity

Let D be a circuit of AND and OR gates (with negations at the inputs). Size(D) = # of wires in D.

Size(f) = min{Size(D) : D computes f} We may allow oracle gates for a set A, along

with AND and OR gates. SizeA(f) = min{Size(D) : DA computes f}


K-complexity ≈ Circuit ComplexityK-complexity ≈ Circuit Complexity

There are some obvious similarities in the definitions. What are some differences?

A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings.

– Given any string x, let fx be the function whose truth table is the string of length 2log|x|, padded out with 0’s, and define Size(x) to be Size(fx).


K-complexity ≈ Circuit ComplexityK-complexity ≈ Circuit Complexity

There are some obvious similarities in the definitions. What are some differences?

A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings.

A more fundamental difference:

– C(x) is not computable; Size(x) is. The Minimum Circuit Size Problem (MCSP):

{(x,i) : Size(x) ≤ i}.


MCSPMCSP

MCSP is in NP, but is not known to be NP-complete.

MCSP is not believed to be in P.

– Factoring is in BPPMCSP.

– Every cryptographically-secure one-way function can be inverted in PMCSP/poly.


So how can K-complexity and Circuit complexity be the same?


C(x) ≈ SizeH(x), where H is the halting problem.

For one direction, let U(d) = x. We need a small circuit (with oracle gates for H) for fx, where fx(i) is the i-th bit of x. This is easy, since {(d,i,b) : U(d) outputs a string whose i-th bit is b} is computably-enumerable.

For the other direction, let SizeH(fx) = m. No oracle gate has more than m wires coming into it. Given a description of D (size not much bigger than m) and the m-bit number giving the size of {y in H : |y| ≤ m}, U can simulate DH and produce fx




C(x) ≈ SizeH(x), where H is the halting problem.

So there is a connection between C(x) and Size(x) …

…but is it useful? First, let’s look at decidable versions of

Kolmogorov complexity.


Time-Bounded Kolmogorov Complexity


The usual definition: Ct(x) = min{|d| : U(d) = x in time t(|d|)}. Problems with this definition

– No invariance! If U and U’ are different universal Turing machines, Ct

U and CtU’ have

no clear relationship.

– (One can bound CtU by Ct’

U’ for t’ slightly larger than t – but nothing can be done for t’=t.)

No nice connection to circuit complexity!




Levin’s definition: Kt(x) = min{|d|+log t : U(d) = x in time t(|d|)}. Invariance holds! If U and U’ are different

universal Turing machines, KtU(x) and KtU’(x) are within log |x| of each other.

Let A be complete for E = Dtime(2O(n)). Then Kt(x) ≈ SizeA(x).




Levin’s definition: Kt(x) = min{|d|+log t : U(d) = x in time t(|d|)}. Why log t?

– This gives an optimal search order for NP search problems.

– Adding t instead of log t would give every string complexity ≥ |x|.

…So let’s look at how to make the run-time be much smaller.


Revised Kolmogorov ComplexityRevised Kolmogorov Complexity

C(x) = min{|d| : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x} (where bit # i+1 of x is *).

– This is identical to the original definition. Kt(x) = min{|d|+log t : for all i ≤ |x| + 1, U(d,i,b)

= 1 iff b is the i-th bit of x, in time t(|d|)}.

– The new and old definitions are within O(log |x|) of each other.

Define KT(x) = min{|d|+t : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}.


Kolmogorov Complexity is Circuit Complexity


C(x) ≈ SizeH(x). Kt(x) ≈ SizeE(x). KT(x) ≈ Size(x). Other measures of complexity can be

captured in this way, too:

– Branching Program Size ≈ KB(x) = min{|d|+2s : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in space s(|d|)}.






– Formula Size ≈ KF(x) = min{|d|+2t : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}, for an alternating Turing machine U.


…but is this interesting?…but is this interesting?

The result that Factoring is in BPPMCSP was first proved by observing that, in PMCSP, one can accept a large set of strings having large KT complexity (and by making use of many important results in the theory of pseudorandom generators and derandomization).

(Basic Idea): There is a pseudorandom generator based on factoring, such that factoring is in BPPT for any test T that distinguishes truly random strings from pseudorandom strings. MCSP is such a test.


This idea has many variants.This idea has many variants.

Consider RKT, RKt, and RC.

RKT is in coNP, and not known to be coNP hard.

RC is not hard for NP under poly-time many-one reductions, unless P=NP.

– How about more powerful reductions?

– Is there anything interesting that we could compute quickly if C were computable, that we can’t already compute quickly?

– Proof uses PRGs, Interactive Proofs, and the fact that an element of RC of length n can be found in

But RC is undecidable! Perhaps H is in P relative to RC?




RKT is in coNP, and not known to be coNP hard.

RC, is not hard for NP under poly-time many-one reductions, unless P=NP.


– PSPACE is in P relative to RC.

– NEXP is in NP relative to RC.

– Proof uses PRGs, Interactive Proofs, and the fact that an element of RC of length n can be found in poly time, relative to RC [BFNV].

But RC is undecidable! Perhaps H is in P relative to RC?


Relationship between H and RCRelationship between H and RC

Perhaps H is in P relative to RC?

This is still open. It is known that there is a computable time bound t such that H is in DTime(t) relative to RC [Kummer].

…but the bound t depends on the choice of U in the definition of C(x).

We also know that H is in P/poly relative to RC.




What about RKt?

RKt, is not hard for NP under poly-time many-one reductions, unless E=NE.


– EXP = NP(RKt).

– RKt is complete for EXP under P/poly reductions.

– Open if RKt is in P!






– Formula Size ≈ KF(x) = min{|d|+2t : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}, for an alternating Turing machine U.






– A similar definition captures depth k threshold circuit size.

[This is the clever transition to start the discussion of circuit lower bounds…]


Big Complexity ClassesBig Complexity Classes

NP P . . NC L (Deterministic Logspace)


TC0 O(1)-Depth Circuits of MAJ gates

AC0 [6] NC1 Log-Depth Circuits

AC0 can’t compute Mod 2 [FSS,A]

AC0 O(1)-Depth Circuits of AND/OR gates

The Main Objects of Interest:Small Complexity Classes




AC0 [6] NC1 Log-Depth Circuits

AC0 can’t compute Mod 2 [FSS,A]

AC0 O(1)-Depth Circuits of AND/OR gates





NC1 Log-Depth Circuits

AC0 [2] can’t compute Mod 3 [R,S]

AC0 [2] AC0 O(1)-Depth Circuits of AND/OR gates




NC1 Log-Depth Circuits


AC0 [6] AC0 [2] AC0 O(1)-Depth Circuits of AND/OR gates




NC1 poly-size formulae


AC0 [6] AC0 [2] AC0 O(1)-Depth Circuits of AND/OR gates




NP has complete sets (under polynomial time reducibility ≤P)

These small classes have complete sets, too (under ≤AC°)

Complete ProblemsComplete Problems


ReductionsReductions

A ≤AC° B means that there is a constant-depth circuit computing A that has the usual AND and OR gates, and also has ‘oracle gates’ for B.

B


NC1

TC0

AC0 [6] AC0 [2] AC0


sorting, multiplication, division

[Naor,Reingold] Pseudorandom Generator


NC1

TC0

AC0 [6] AC0 [2] AC0


BFE: Balanced Boolean Formula Evaluation (AND,OR,XOR)

Word problem over S5


The Word Problem Over S5The Word Problem Over S5

A regular set complete for NC1

=


Complexity Classes are not Invented – They’re Discovered


NP (SAT, Clique, TSP,…) P (Linear Programming, CVP, …) NL (Connectivity, Shortest Paths, 2SAT, …) L (Undirected Connectivity, Acyclicity, …) NC1 (BFE, Regular Sets)

TC0 (Sorting, Multiplication, Division)

We’re interested in NC1 (for instance) not because we want to build formulae for

these functions…






… but because we want to know if the blocks of this partition are distinct.






These classes are real.

They’re important.


Longstanding Open ProblemsLongstanding Open Problems

Is P = NP? Is AC0[6] = NP? Is depth 3 AC0[6] = NP?

We’ll focus on questions such as:

Is BFE in TC0?

Is BFE in AC0[6]?


How Close Are We to Proving Circuit Lower Bounds?


Conventional Wisdom: Not Close At All! No new superpolynomial size lower bounds in

over two decades. Razborov and Rudich: Any “natural” argument

proving a lower bound against a circuit class C yields a proof that C can’t compute a pseudorandom function generator.

Since the [Naor, Reingold] generator is computable in TC0, this is bad news.


More Modest GoalsMore Modest Goals

Problems requiring formulae of size n3 [Håstad]

Problems requiring branching programs of size nearly n loglog n [Beame, Saks, Sun, Vee]

Problems requiring depth d TC0 circuits of size n1+c [Impagliazzo, Paturi, Saks]

Time-Space Tradeoffs [Fortnow, Lipton, Van Melkebeek, Viglas]

There is little feeling that these results bring us any closer to separating complexity classes.


How close are the following two statements?

TC0 Circuits for BFE must be of size n1+Ω(1)

For some c>0, TC0 Circuits for BFE must be of size n1+c.




How close are the following two statements?

TC0 Circuits for BFE must be of size n1+Ω(1)

For some c>0, FTC0 Circuits for BFE must be of size n1+c



This is known [IPS’97]

This implies TC0 ≠ NC1 [A, Koucky]


Self-ReducibilitySelf-Reducibility

A set B is said to be “self-reducible” if B ≤P B



A set B is said to be “self-reducible” if B ≤P B via a reduction that, on input x, does not ask about whether x is in B.

Very well-studied notion. For example, φ is in SAT if and only if

(φ0 is in SAT) or (φ1 is in SAT)



Many of the important problems in (or near) NC1 have a special self-reducibility property:



Many of the important problems in (or near) NC1 have a special self-reducibility property: Instances of length n are AC0-Turing (or TC0-Turing) reducible to instances of length n½ via reductions of linear size.

Examples:

– BFE

– the word problem over S5

– MAJORITY

– Iterated Product of 3-by-3 Integer Matrices


Self ReducibilitySelf Reducibility

BFE

A subformula near the root

Subformulae near inputs



S5



The self-reduction of S5, on inputs of size n, uses (n½ + 1) oracle gates of size n½.

Thus if S5 has TC0 circuits of size nk, it also has circuits of size (n½ + 1)nk/2= O(n(k+1)/2).

Similar arguments hold for other classes (such as AC0[6] and NC1).

More complicated self-reductions can be presented for MAJORITY and Iterated Product of 3-by-3 matrices.


A CorollaryA Corollary

If BFE has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size.

If S5 has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size.

If MAJ has AC0[6] circuits, then it has such circuits of nearly linear size. (Etc.)

Thus, e.g., to separate NC1 from TC0, it suffices to show that BFE requires TC0 circuits of size n1.0000001.


A CorollaryA Corollary

If BFE has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size.

If S5 has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size.

If MAJ has AC0[6] circuits, then it has such circuits of nearly linear size. (Etc.)

How widespread is this phenomenon? Is it true for SAT? (I.e., can we show NP ≠ TC0 by proving that SAT requires TC0 circuits of size n1.0000001?)


Limitations of Self-ReducibilityLimitations of Self-Reducibility

Any problem for which instances of length n are TC0-Turing reducible to instances of length n½ via poly-size reductions lies in NC.

Thus there is no obvious way to apply these techniques to SAT or to problems complete for P.

…but perhaps, rather than showing directly that SAT has this strong form of self-reducibility, one can argue that if SAT is in TC0 then it has TC0 circuits of nearly-linear size.




d levels of oracle gates




d2 levels of oracle gates




d3 levels of oracle gates

After log log rounds,

the depth is logO(1)n


Prospects for ProgressProspects for Progress

We have seen that existing techniques prove bounds that are “nearly” good enough to separate NC1 and TC0. Some of these proofs are “natural”.

Don’t the results of [Razborov & Rudich] indicate that further progress will require very different approaches?

Not necessarily!



The [Razborov & Rudich] framework of natural proofs assumes that a “natural” proof of a lower bound will make use of a combinatorial property that (among other things) is shared by a large fraction of the functions on n bits.

In contrast, we are making use of a self-reducibility property that allows us to boost a n1+ε lower bound to a superpolynomial lower bound. This self-reducibility property holds for only a vanishingly small fraction of all functions.



These observations are simple, but … they have forever changed the way that we

look at quadratic (and smaller) lower bounds. We are not claiming to have found a way

around the obstacles identified by [Razborov & Rudich]. (Such a claim will have to wait until someone proves that NC1 ≠ TC0.) But we do believe that this avenue deserves further exploration.


There are good reasons to develop and explore the connections between Kolmogorov complexity and circuit complexity.

There are many open problems in this area that I will be delighted to discuss with you in more detail.

There are two bad typos in the proceedings version of the paper. (“P” should be “NP”.) A corrected version is available at my home page.

ConclusionConclusion


Connections between Kolmogorov Complexity and Circuit Complexity might be relevant to the question of whether NEXP is contained in (non-uniform) TC0 (depth 3).

SpeculationSpeculation


[IKW] showed that NEXP is in P/poly iff NEXP = MA iff MA cannot be derandomized

The proof shows that NEXP is in P/poly iff every set in P contains strings of KT-complexity O(log n) iff NEXP = IP[P/poly].



Similar techniques show: NEXP is in nonuniform NC1

iff every set in P contains strings of KF-complexity O(log n) iff NEXP = MIPNC1 iff MIPNC1 cannot be derandomized.

NEXP is in nonuniform TC0 iff every set in P contains strings of small complexity iff NEXP = MIPTC0 iff MIPTC0 cannot be derandomized.



What else happens in such a collapse? If NP = uniform TC0, then #P is not contained in

non-uniform TC0 (so NEXP is not in non-uniform TC0).

So let’s consider NEXP = MIPTC0 and NP ≠ uniform TC0. If this “hardness assumption” were sufficient to “derandomize” MIPTC0 then this would give the desired lower bound on NEXP…

[Fortnow, Klivans], [van Melkebeek, Santhanam]


eric allender rutgers university circuit complexity, kolmogorov complexity, and prospects for lower...

Documents