eric allender rutgers university the audacity of computational complexity theory

Eric AllenderRutgers University

The Audacity of Computational Complexity Theory

The Audacity of Computational Complexity Theory

Eric Allender: The Audacity of Computational Complexity Theory < 2 >

Today’s MessageToday’s Message

Computational complexity theory is successful because of some audacious notions.


The Case against Complexity TheoryThe Case against Complexity Theory

Exhibit #1: The P vs NP question.

– If you’ve heard anything at all about computational complexity theory, this is what you’ve heard about.


Polynomial time

– Time n100 is not feasible.

Turing Machines

– Clearly an unrealistic model, even for deterministic machines.

Asymptotic Analysis

– An example will illustrate an important point.

Three Absurd Notions in Complexity Theory

Three Absurd Notions in Complexity Theory


An example: the Game of GoAn example: the Game of Go

Computing strategies for Go requires exponential time.

– More precisely, given an n-by-n Go board with tiles on it, no program can compute an optimal next move in fewer than c2n – d steps, for some constants c and d.

– Thus any program solving this problem must run very slowly on large inputs. This is the essence of asymptotic analysis.





– This is a much stronger statement about complexity than we are able to prove for most problems.





– Conceivably, there is a linear-time program that computes optimal moves, even for Go boards of size 1000-by-1000!


ConclusionsConclusions

Complexity Theory is a failure. For most problems of interest, no proofs of

infeasibility are known. Even for problems where we can prove that

exponential time is required (such as Go), we can’t conclude anything about input sizes that are actually of practical interest.

Computational complexity theory is a huge waste of time, and we should all go home now.


Wrong ConclusionsWrong Conclusions

Complexity Theory is a failure. For most problems of interest, no proofs of

infeasibility are known. Even for problems where we can prove that

exponential time is required (such as Go), we can’t conclude anything about input sizes that are actually of practical interest.

Computational complexity theory is a huge waste of time, and we should all go home now.


An Example of the Type of Theorem that We Need

An Example of the Type of Theorem that We Need

Theorem: Any circuit that takes as input a logical formula (in WS1S) of length 616 and produces as output a correct answer, saying if the formula is valid or not, has at least 10123 gates. (Stockmeyer, 1974)

Corollary: No program can run on any computer existing today, and solve this problem with a running time of less than 1000 years.


Possible ObjectionsPossible Objections


Is this a problem anyone would ever want to compute? Yes! It’s used in hardware

verification.




Isn’t this technology-dependent? Slightly. It’s based on AND and

OR gates.




Aha!! What about quantum computers? Essentially the same result holds.

Some of the constants will be slightly different.


No Possible Objections!No Possible Objections!


Although this theorem is about finite functions, the proof is in terms of functions on infinite domains. (That is, it uses asymptotic analysis.)


Two Parts to the ProofTwo Parts to the Proof

Lemma: There is a problem A computable in space 2n that requires circuits of size nearly 2n.

– Diagonalization is good for creating monsters.

Small circuits for the validity problem yield small circuits for A.

– ReducibilityAn easy-to-compute function f such that

x is in A if and only if

f(x) is a valid formula


Two Parts to the ProofTwo Parts to the Proof



Small circuits for the validity problem yield small circuits for A.

– Every problem computable in space 2n is reducible to the validity problem.


Reductions: A Reason to CelebrateReductions: A Reason to Celebrate

Reducibility is a shockingly effective tool for understanding the complexity of real-world computational problems.

One reason that we are here today, is to celebrate the discovery of this fact.

Let’s learn more about reductions.


Desirable Properties for ReductionsDesirable Properties for Reductions

Reductions should be “easy” functions. If f and g are easy to compute, then it should

also be easy to compute f(g(x)). If f is computable in time n2, then f is easy.

But this implies that there are “easy” functions that time n100,000

to compute!


Desirable Properties for ReductionsDesirable Properties for Reductions

Reductions should be “easy” functions. If f and g are easy to compute, then it should

also be easy to compute f(g(x)). If f is computable in time n2, then f is easy.

We have two choices:

1. Forget about our properties above, or

2. Embrace them, and claim that they make sense.


A Justification for Polynomial TimeA Justification for Polynomial Time

Complexity theoreticians aren’t trying to show that things are easy; they’re trying to show that things are hard.

If a problem cannot be solved in polynomial time, then this is a strong indication that it is not feasible to compute.

A “Karp reduction” is a function f computable in polynomial time.


Karp ReductionsKarp Reductions

Recall that if f reduces A to B, then A is “no harder than” B.

Say that A and B are “equivalent” if A is reducible to B and vice-versa.

Equivalent problems have roughly the same complexity. They are two different ways of looking at the same problem.

Hundreds of computational problems have been studied. Surprisingly, they fall into a very small number of equivalence classes.


Checkers & GoCheckers & Go

Checkers and Go are equivalent. Fact: Optimal strategies can be computed in

time approximately 2n. Fact: Every problem computable in time 2poly(n)

is reducible to Go. Hence this is a very special equivalence class.

It consists of all of the “hardest” problems in EXP (the class of problems computable in exponential time).


Checkers & GoCheckers & Go

Checkers and Go are equivalent. Fact: Optimal strategies can be computed in

time approximately 2n. Fact: Every problem computable in time 2poly(n)

is reducible to Go. We say these are “complete for EXP”. Since some problems in EXP require time 2n,

we know that the complete problems also require (nearly) this much time.


CompletenessCompleteness

Fact: Another big equivalence class is complete for PSPACE (the class of problems that can be computed using a polynomial amount of memory).

If any problem in PSPACE requires exponential time to compute, then all of these PSPACE-complete problems do.


CompletenessCompleteness

Fact: Another big equivalence class is complete for PSPACE (the class of problems that can be computed using a polynomial amount of memory).

A quick program solving any PSPACE-complete problem gives a general technique to speed up any memory-limited computation.

Thus, we guess that all PSPACE-complete problems require (nearly) exponential time.


Completeness, continuedCompleteness, continued

Some equivalence classes correspond to time and space complexity.

…but most don’t! …and they are the classes with the problems

we really care the most about.


NondeterminismNondeterminism

The solution: Consider weird machines. Given an input x, a nondeterministic machine

is allowed to search for free through an exponentially-large set of strings, looking for a proof that x should be accepted.

You may object: “This is absurd. No machine like this can be built!”

…That’s the point!


NPNP

NP is the class of problems solvable in polynomial time on nondeterministic machines.

A huge number of important computational problems are NP-complete:

– The traveling salesman problem

– The 3-colorability problem

– Partition

– Knapsack …

Nondeterminism is forced on us.


coNPcoNP

What is the relationship between the following two problems?– 3-colorability – Non-3-colorability

They have the same (deterministic) time complexity, but they seem not to be in the same Karp-equivalence class.

There are short proofs that a graph can be three-colored.

– Simply present the coloring! Is there always a short proof that a graph

cannot be three-colored?


coNPcoNP

What is the relationship between the following two problems?– 3-colorability – Non-3-colorability

They have the same (deterministic) time complexity, but they seem not to be in the same Karp-equivalence class.

coNP consists of the complements of problems in NP.

Nondeterminism highlights differences among problems of equivalent deterministic complexity.


Other Equivalence ClassesOther Equivalence Classes

Many other equivalence classes of important problems are captured in terms of time- and space-bounds on variants of nondeterministic machines.

Any two problems in P are in the same Karp-equivalence class.

– But using more restricted reductions, a similar picture emerges inside P.

– Amazingly, even with restricted reductions, the classes of complete sets for “big” complexity classes (EXP, NP, …) are essentially unchanged.


Before ReducibilityBefore Reducibility

There was almost nothing that we could say about the computational complexity of real-world computational problems.

For evidence of intractability, one could only point to the fact that other people had tried, and failed, to find an efficient algorithm.


With ReducibilityWith Reducibility

Order is imposed on the chaos of problems. Natural computational problems fall into a

handful of equivalence classes. We have a reason to believe that these

classes are, in fact, distinct.

– Time- and Space-bounded complexity classes (on different types of machines)

A theory that explains perceived differences in computational difficulty.


FactoringFactoring

A good illustration of how far we’ve come. Complexity theory has little to say about the

factorization problem

– Some cryptographic problems are “reducible” to factoring.

– There is no machine model or complexity class for which factoring is complete.

The only real reason to think that factoring is hard, is because smart people have failed to find fast algorithms.


FactoringFactoring



– RSA is “reducible” to factoring.


This is the situation we had for almost all computational problems, before the 70’s.


FactoringFactoring



– RSA is “reducible” to factoring.


Now, the reverse holds. For almost all problems that we suspect to be difficult to compute, we can “explain” why they are hard.


…but what about the problem of asymptotic analysis???

…but what about the problem of asymptotic analysis???

Recall that, although Go and other EXP-complete problems require exponential time, we couldn’t say anything about fixed input lengths.

In contrast, we could say something about the validity problem for inputs of length 616.


Two Parts to the Proof that Validity is Hard for Inputs of size 616:




Everything computable in space 2n is reducible to the validity problem.

Can we mimic this, to show a similar result for Go?






Everything computable in space 2n is reducible to the validity problem.



An Attempt to Prove a Similar Infeasibility Theorem for Go.

An Attempt to Prove a Similar Infeasibility Theorem for Go.

Lemma: There is a problem A computable in sitime 2n that requires circuits of size nearly 2n. …..

– We don’t know how to prove this! is good for creating monsters.

Everything computable in sitime 2n is reducible to the problem of finding optimal moves in Go.

– This part goes through with no problem!



A Central Question in Complexity Theory:


Does every problem in EXP have small circuits?

– Is EXP contained in P/poly?

It would be very strange if this were true!

If NP requires large circuits, then it should be possible to prove infeasibility results for fixed input lengths for NP-complete problems.




Does every problem in EXP have small circuits?

– Is EXP contained in P/poly?

It would be very strange if this were true!

In the next talk, Avi Wigderson will present more reasons why the circuit complexity of EXP is of such importance.



Complexity theory does provide convincing proofs that certain transformations from input to output cannot be computed.

In order to prove that finite functions are hard to compute, we use a theoretical framework based on infinite functions.

This theoretical framework can be used to provide evidence of infeasibility, even when we cannot prove that functions are hard to compute.



Reducibility is unreasonably effective in characterizing the complexity of problems.

With surprisingly few exceptions, most problems are complete for one of a handful of complexity classes.

A lot of exciting progress is being made. The next two talks will discuss some more recent developments.

eric allender rutgers university the audacity of computational complexity theory

Documents

complexity theory exhibit

polynomial time time

exponential time

n d steps

lineartime program

huge waste of time

constants c

computing strategies