upper bounds on k -sat

9/23/04 1

Upper bounds onUpper bounds onkk-SAT-SAT

Chris CalabroChris [email protected]@cs.ucsd.edu

9/23/04 2

What is What is kk-SAT?-SAT? Variables – xVariables – x11, x, x22, x, x33, …, … Literals – xLiterals – x11, ¬x, ¬x11, x, x22, ¬x, ¬x22,…,… Clauses – (xClauses – (x11 or ¬x or ¬x22 or x or x33), (x), (x22), ()), () CNFs – (xCNFs – (x11 or ¬x or ¬x22 or x or x33) and (x) and (x22)) k-CNFs – (xk-CNFs – (x11 or ¬x or ¬x22 or x or x33) and (x) and (x22))

k

_____________

9/23/04 3

AssignmentAssignment (x(x11 or ¬x or ¬x22 or x or x33) and (x) and (x22))

9/23/04 4

AssignmentAssignment (x(x11 or or ¬x¬x22 or x or x33) and (x) and (x22))

xx11 xx22 xx33

00 00 00

9/23/04 5

AssignmentAssignment ((xx11 or ¬x or ¬x22 or or xx33) and () and (xx22))

xx11 xx22 xx33

11 11 11

9/23/04 6

Formal problemFormal problem Decision version – k-SAT={F Decision version – k-SAT={F εε k- k-

CNF | exists assignment a with CNF | exists assignment a with F(a)=1}F(a)=1}

Search version – given F Search version – given F εε k-CNF k-CNF find an assignment a with F(a)=1find an assignment a with F(a)=1 or output “not satisfiable”or output “not satisfiable”

k-SAT k-SAT εε NPC NPC

9/23/04 7

OutlineOutline MotivationMotivation

Why care about NPC problems?Why care about NPC problems? Why upper bounds?Why upper bounds? Why SAT?Why SAT? How hard is SAT?How hard is SAT?

AlgorithmsAlgorithms DesignDesign ConclusionConclusion

9/23/04 8

NPCNPC Language L Language L εε NPC iff NPC iff

L L εε NP NP forall A forall A εε NP, A ≤p L NP, A ≤p L

VERIFY={<M,x> | M is a Turing machine, x VERIFY={<M,x> | M is a Turing machine, x εε ΣΣ**, exists w , exists w εε ΣΣ|x||x|, M(x,w) accepts in time |x|} , M(x,w) accepts in time |x|} εε NPCNPC

Solve L Solve L εε NPC efficiently => forall A NPC efficiently => forall A εε NP, NP, solve A efficientlysolve A efficiently

forall L forall L εε NPC, (L NPC, (L εε P iff P=NP) P iff P=NP)

9/23/04 9

The big questionThe big question P = NP?P = NP? Posed by Kurt Gödel to John von Posed by Kurt Gödel to John von

Neumann in 1956Neumann in 1956 Almost the same question: NP ≤ Almost the same question: NP ≤

BPP?BPP?

9/23/04 10

Who cares?Who cares? Complexity theoristsComplexity theorists CryptographersCryptographers EngineersEngineers Everybody else!Everybody else!

9/23/04 11

Possible worldsPossible worlds P=NP? UnresolvedP=NP? Unresolved

Theoreticians need to keep saying, “unless Theoreticians need to keep saying, “unless P=NP”P=NP”

Could be wasting time on pointless theoryCould be wasting time on pointless theory Can’t fully trust current crypto systemsCan’t fully trust current crypto systems

P=NPP=NP Poly hierarchy collapses: for each Poly hierarchy collapses: for each fixedfixed m, m,

can answer questions of formcan answer questions of form

in poly timein poly time

),,( 121 mm xxxxx

9/23/04 12

If SAT If SAT εε DTIME(n DTIME(nkk), k ), k largelarge

Crypto Crypto theorytheory changes changes No OWFsNo OWFs No trapdoor functionsNo trapdoor functions

Crypto Crypto practicepractice? Could there be a ? Could there be a more efficient algorithm out there?more efficient algorithm out there? PRIMES PRIMES εε DTIME(n DTIME(n1212) [AKS02]) [AKS02] PRIMES PRIMES εε RTIME(n RTIME(n33) [M76,R80]) [M76,R80] What about BQP?What about BQP?

9/23/04 13

If SAT If SAT εε DTIME(n DTIME(nkk), k ), k smallsmall

No RSA, Diffie-Hellman, DES, AES, No RSA, Diffie-Hellman, DES, AES, SHA1SHA1

Symmetric key crypto not Symmetric key crypto not necessarily doomednecessarily doomed Info theoretically secure key exchange Info theoretically secure key exchange

in bounded-storage model [AR99]in bounded-storage model [AR99] Quantum key exchange [BB84]Quantum key exchange [BB84]

9/23/04 14

Secure symmetric Secure symmetric encryptionencryption

9/23/04 15

The good stuffThe good stuff Test circuits/programs for Test circuits/programs for

correctnesscorrectness Automatically generate proofs of Automatically generate proofs of

mathematical theoremsmathematical theorems Build better machinesBuild better machines

Faster CPUFaster CPU Perfect airplane wingPerfect airplane wing

Solve AI problemsSolve AI problems

9/23/04 16

Black-box mistake-bounded Black-box mistake-bounded learninglearning

Online problem: at time tOnline problem: at time t Input – (xInput – (x11,y,y11),…,(x),…,(xtt,y,ytt) where |x) where |xii|=n, |y|=n, |yii||

=1 coming from circuit C=1 coming from circuit C Output – circuit COutput – circuit Ctt Goal – CGoal – Ctt equivalent to C equivalent to C

Can bound # times t with CCan bound # times t with Ctt(x(xt+1t+1)≠y)≠yt+1t+1 by O(|C|by O(|C|2 2 lg|C|)lg|C|)

If |C| can be estimated, O(|C| lg|C|)If |C| can be estimated, O(|C| lg|C|)

9/23/04 17

Why upper bounds?Why upper bounds? Most researchers (61 to 9) think Most researchers (61 to 9) think

P≠NP [Gas02]P≠NP [Gas02] Why look for good upper bounds? Why look for good upper bounds?

There aren’t any, right?There aren’t any, right?

NP problems could be easy in NP problems could be easy in average caseaverage case

There could be practical algorithmsThere could be practical algorithms

9/23/04 18

Why SAT?Why SAT? Why not subset sum, vertex cover, Why not subset sum, vertex cover,

hampath, TSP?hampath, TSP? Formulas look like natural language Formulas look like natural language

questionsquestions Common problems Common problems efficientlyefficiently reduce reduce

to SAT, often preserving structure of to SAT, often preserving structure of solution spacesolution space

Many practical algorithms for SATMany practical algorithms for SAT

9/23/04 19

An applicationAn application Circuit equivalence checkingCircuit equivalence checking

Standard techniques – random simulation, Standard techniques – random simulation, BDDsBDDs

Alternative – reduce to SATAlternative – reduce to SAT

[BCCFZ99] compared performance on [BCCFZ99] compared performance on real circuits with 1000s of gatesreal circuits with 1000s of gates

BDDsBDDs SATOSATO> 1 day> 1 day 1 min – 1 1 min – 1

hh

9/23/04 20

How hard is SAT?How hard is SAT? DefineDefine

sskk=inf{=inf{εε>0 | exists random alg A solving k->0 | exists random alg A solving k-SAT in time O(2SAT in time O(2εεnn)})}

σσkk=inf{=inf{εε>0 | exists random alg A solving >0 | exists random alg A solving uniqueunique k-SAT in time O(2 k-SAT in time O(2εεnn)})}

ss∞∞=lim=limk→∞k→∞ s skk σσ∞∞=lim=limk→∞k→∞ σσkk

Exponential time hypothesis (ETH) – sExponential time hypothesis (ETH) – s33>0>0forall k≥3 (ETH↔sforall k≥3 (ETH↔skk>0)>0)

9/23/04 21

Upper bounds on sUpper bounds on skk

kk 33 44 ∞∞

Upper Upper boundbound

.415037 .415037 [Sch99][Sch99]

.75 [PPZ97].75 [PPZ97] 11

.411706 .411706 [SSW01][SSW01]

.58496 .58496 [Sch99][Sch99]

.411635 .411635 [HSSW02][HSSW02]

.56249 .56249 [PPSZ03][PPSZ03]

.411112 .411112 [Rol03][Rol03]

9/23/04 22

OutlineOutline MotivationMotivation

Why care about NPC problems?Why care about NPC problems? Why upper bounds?Why upper bounds? Why SAT?Why SAT? How hard is SAT?How hard is SAT?

AlgorithmsAlgorithms DesignDesign ConclusionConclusion

9/23/04 23

OutlineOutline MotivationMotivation AlgorithmsAlgorithms

Problem variantsProblem variants Incremental assignmentIncremental assignment Local searchLocal search HeuristicsHeuristics

DesignDesign ConclusionConclusion

9/23/04 24

Problem variantsProblem variants General k-SAT – worst caseGeneral k-SAT – worst case Unique k-SAT – promise problemUnique k-SAT – promise problem

YES – exactly 1 solutionYES – exactly 1 solution NO – 0 solutionsNO – 0 solutions

Random k-SATRandom k-SAT Choose m random k-clauses with replacementChoose m random k-clauses with replacement Plant a solution – choose random assignment a, then Plant a solution – choose random assignment a, then

choose m clauses that agree with achoose m clauses that agree with a k-SAT from “real world” problemk-SAT from “real world” problem

Bounded model checkingBounded model checking Circuit fault analysisCircuit fault analysis PlanningPlanning

9/23/04 25

Algorithmic paradigmsAlgorithmic paradigms Incremental assignment (DPLL) – Incremental assignment (DPLL) –

assign variables 1 at a time, assign variables 1 at a time, simplifying formula as you gosimplifying formula as you go

Local search – choose an initial Local search – choose an initial assignment, perform random walk assignment, perform random walk directed by formuladirected by formula

Heuristics – combine features of Heuristics – combine features of other algorithms ad hocother algorithms ad hoc

9/23/04 26

Incremental assignmentIncremental assignmentOrdered-DLL(F)Ordered-DLL(F)

do t timesdo t timesfor i=1,…,nfor i=1,…,nchoose unassigned var x at randomchoose unassigned var x at randomif clause {x} if clause {x} εε F, assign x ← 1 F, assign x ← 1else if {¬x} else if {¬x} εε F, assign x ← 0 F, assign x ← 0else assign x at randomelse assign x at randomsimplify F by removing true clauses, false simplify F by removing true clauses, false literalsliteralsif F=1, return assignmentif F=1, return assignment

9/23/04 27

Incremental assignment - Incremental assignment - analysisanalysis

Idea – lower bound probability p that each Idea – lower bound probability p that each variable gets forcedvariable gets forced

DefinitionDefinition Clause C Clause C εε F is F is criticalcritical at solution a iff C has at solution a iff C has

exactly 1 true literal at aexactly 1 true literal at a Solution a is Solution a is isolatedisolated iff flipping any variable in a iff flipping any variable in a

yields a nonsolutionyields a nonsolution Claim – a isolated => F has ≥ 1 critical clause Claim – a isolated => F has ≥ 1 critical clause

for each variable => p ≥ 1/k, conditioned on for each variable => p ≥ 1/k, conditioned on making random assignments in agreement making random assignments in agreement with awith a

9/23/04 28

Incremental assignment - Incremental assignment - analysisanalysis

Corollary – a isolated => Pr(find a in 1 Corollary – a isolated => Pr(find a in 1 iteration) ≥ 2iteration) ≥ 2-(1-1/k)n-(1-1/k)n

Either there is a nearly isolated Either there is a nearly isolated solution or there are many solutions solution or there are many solutions => running time ≤ poly(n)2=> running time ≤ poly(n)2(1-1/k)n(1-1/k)n [PPZ97][PPZ97]

Strengthened analysis – poly(n)(2Strengthened analysis – poly(n)(2nn/s)/s)1-1-

1/k1/k [CIKP03] [CIKP03]

9/23/04 29

ResolutionResolution Each clause provides info about Each clause provides info about

solution setsolution set Why not add preprocessing step that Why not add preprocessing step that

adds clauses implied by formula F?adds clauses implied by formula F?

(x(x11 or or αα) and (¬x) and (¬x11 or or ββ) => ) => αα or or ββ Definition – F is Definition – F is fully resolvedfully resolved iff F is iff F is

closed under resolutionclosed under resolution

9/23/04 30

Resolution is completeResolution is complete Let S = solution space of formula FLet S = solution space of formula F Definition – variable x is Definition – variable x is determineddetermined iff iff

forall a forall a εε S, a(x)=1 S, a(x)=1 or forall a or forall a εε S, a(x)=0 S, a(x)=0

Theorem – let F be fully resolved, x be Theorem – let F be fully resolved, x be a variable, b a variable, b εε {0,1}, then {0,1}, then

F is satisfiable iff ¬(Ø F is satisfiable iff ¬(Ø εε F) F) x is determined iff F contains a unit clause x is determined iff F contains a unit clause

in xin x F|F|x←bx←b is fully resolved is fully resolved

9/23/04 31

Bounded resolutionBounded resolution Problems with full resolutionProblems with full resolution

Resolvent of 2 k-clauses could have size 2k-Resolvent of 2 k-clauses could have size 2k-22

Exponentially many large clausesExponentially many large clauses Shortest resolution refutation of Shortest resolution refutation of

unsatisfiable formulas may be exponentially unsatisfiable formulas may be exponentially largelarge

Simple solution – s-bounded resolutionSimple solution – s-bounded resolution For some constant s, produce resolvent R For some constant s, produce resolvent R

only if |R| ≤ sonly if |R| ≤ s

9/23/04 32

Resolve SATResolve SAT Algorithm – preprocess F by performing s-Algorithm – preprocess F by performing s-

bounded resolution (s=O(lg n)), then use bounded resolution (s=O(lg n)), then use ordered-DLLordered-DLL

Bound improves from poly(n)2Bound improves from poly(n)2(1-1/k)n(1-1/k)n to to poly(n)2poly(n)2(1-(1-μμkk/(k-1))n/(k-1))n where where , k≥5, k≥5

Same bound for unique k-SAT, k≥3Same bound for unique k-SAT, k≥3

Best k-SAT alg for k≥4. O(2Best k-SAT alg for k≥4. O(2.5625n.5625n) for k=4.) for k=4. Best unique k-SAT alg for k≥3. O(2Best unique k-SAT alg for k≥3. O(2.3863n.3863n) for ) for

k=3.k=3.

1 11 )(

1j k

k jj

9/23/04 33

Local searchLocal searchLocal-search(F)Local-search(F)

do t timesdo t timeschoose random assignment achoose random assignment ado 3n timesdo 3n timesif F(a)=1, return aif F(a)=1, return achoose an unsatisfied clause Cchoose an unsatisfied clause Cchoose var x in C at randomchoose var x in C at randomflip x in aflip x in a

9/23/04 34

Local search - analysisLocal search - analysis Let z be a particular solutionLet z be a particular solution Can model each iteration as a Can model each iteration as a

Markov processMarkov process States – 0,1,2,…States – 0,1,2,… State i represents dist(a,z)=iState i represents dist(a,z)=i State 0 is absorbingState 0 is absorbing Initial state is binomial (n,1/2), then Initial state is binomial (n,1/2), then

algorithm performs a random walkalgorithm performs a random walk

9/23/04 35

Local search - analysisLocal search - analysis Let z be a particular solutionLet z be a particular solution Can model each iteration as a Markov Can model each iteration as a Markov

processprocess States – 0,1,2,…States – 0,1,2,… State i represents d(a,z)=IState i represents d(a,z)=I State 0 is absorbingState 0 is absorbing Initial state is binomial (n,1/2), then Initial state is binomial (n,1/2), then


0---1---2---0---1---2---33---4---5---6---7---…---4---5---6---7---…

9/23/04 36




0---1---0---1---22---3---4---5---6---7---…---3---4---5---6---7---…

9/23/04 37




0---1---2---0---1---2---33---4---5---6---7---…---4---5---6---7---…

9/23/04 38




0---1---0---1---22---3---4---5---6---7---…---3---4---5---6---7---…

9/23/04 39




0---0---11---2---3---4---5---6---7---…---2---3---4---5---6---7---…

9/23/04 40




0---1---0---1---22---3---4---5---6---7---…---3---4---5---6---7---…

9/23/04 41




0---0---11---2---3---4---5---6---7---…---2---3---4---5---6---7---…

9/23/04 42




00---1---2---3---4---5---6---7---…---1---2---3---4---5---6---7---…

9/23/04 43

Local search - analysisLocal search - analysis Pr(go left) ≥ 1/kPr(go left) ≥ 1/k Pr(go right) ≤ 1-1/kPr(go right) ≤ 1-1/k Let p = Pr(reach 0 in one iteration)Let p = Pr(reach 0 in one iteration)

Naïve analysis – p ≥Naïve analysis – p ≥Pr(reach 0 by successive left moves) ≥ Pr(reach 0 by successive left moves) ≥

Better analysis – use reflection principle to Better analysis – use reflection principle to showshowp ≥ p ≥

Running time – Running time – [Sch99][Sch99]

nk ))1(( 1

21

nknpoly ))1(()( 11

211

nknpoly ))1(2)(( 1

9/23/04 44

HeuristicsHeuristics Complete – gives correct answer with Complete – gives correct answer with

probability 1 (satz, SATO, zChaff)probability 1 (satz, SATO, zChaff) Incomplete – sometimes fails to find Incomplete – sometimes fails to find

witnesswitness Probabilistically Approximately Complete Probabilistically Approximately Complete

(PAC) – succeed with probability 1 without (PAC) – succeed with probability 1 without restarts if given enough time (UnitWalk)restarts if given enough time (UnitWalk)

Essentially incomplete – succeed with Essentially incomplete – succeed with probability <1 unless restarted (GSAT, probability <1 unless restarted (GSAT, GWSAT, HSAT, HWSAT, WalkSAT)GWSAT, HSAT, HWSAT, WalkSAT)

9/23/04 45

UnitWalkUnitWalkchoose initial assignment achoose initial assignment ado t timesdo t times

use ordered-DLL (with a as use ordered-DLL (with a as randomness!) to find next randomness!) to find next assignmentassignment

9/23/04 46

VariantsVariants UnitWalk+WalkSAT – alternate steps of UnitWalk+WalkSAT – alternate steps of

UnitWalk and WalkSAT!UnitWalk and WalkSAT! UnitWalk+incBinSAT+WalkSAT – add UnitWalk+incBinSAT+WalkSAT – add

preprocessing step of 2-bounded resolutionpreprocessing step of 2-bounded resolution

Common benchmarksCommon benchmarks Random formulas, circuit problems, bounded-Random formulas, circuit problems, bounded-

model checking, planning problemsmodel checking, planning problems Variables and clauses – 100s to 10,000sVariables and clauses – 100s to 10,000s Running times – seconds to minutesRunning times – seconds to minutes

9/23/04 47

OutlineOutline MotivationMotivation AlgorithmsAlgorithms

Problem variantsProblem variants Incremental assignmentIncremental assignment Local searchLocal search HeuristicsHeuristics

DesignDesign ConclusionConclusion

9/23/04 48

OutlineOutline MotivationMotivation AlgorithmsAlgorithms DesignDesign

SparsificationSparsification IsolationIsolation

ConclusionConclusion

9/23/04 49

SparsificationSparsification Given k-CNF F, Given k-CNF F, εε > 0, > 0,

we can generate in time poly(n)2we can generate in time poly(n)2εεnn a a disjunction G of 2disjunction G of 2εεnn k-CNFs G k-CNFs Gii, each of size , each of size O(n) [IPZ98]O(n) [IPZ98]

Each GEach Gii can be produced in time poly(n) can be produced in time poly(n)

Can be used as a preprocessing step whenCan be used as a preprocessing step when Algorithm is exponential time anywayAlgorithm is exponential time anyway Linear sized formulas are neededLinear sized formulas are needed

9/23/04 50

Sparsification exampleSparsification example Theorem – Theorem – σσ33 = 0 => s = 0 => s∞∞ = 0 = 0

[CIKP03][CIKP03] Proof shows a reduction from general Proof shows a reduction from general

k-SAT to unique 3-SATk-SAT to unique 3-SAT To do this requires clause width reductionTo do this requires clause width reduction Standard algorithm produces formula Standard algorithm produces formula

with O(n+m) variables (m = #clauses in with O(n+m) variables (m = #clauses in input formula)input formula)

Sparsify first! Then O(n+m)=O(n)Sparsify first! Then O(n+m)=O(n)

9/23/04 51

IsolationIsolation Reducing general k-SAT to unique k-Reducing general k-SAT to unique k-

SAT [VV86]SAT [VV86] Intersect solution space with O(n) linear Intersect solution space with O(n) linear

equations in GF(2)equations in GF(2) Reduces the number of solutions to 1 Reduces the number of solutions to 1

with inverse polynomial success with inverse polynomial success probabilityprobability

Need to Need to squaresquare number of variables to number of variables to represent such equationsrepresent such equations

9/23/04 52

IsolationIsolation SolutionSolution

use use sparsesparse linear equations ( linear equations (εεn variables each) to n variables each) to constrain solutions to a small Hamming ball (radius constrain solutions to a small Hamming ball (radius εεn)n)

Use random restriction – set O(Use random restriction – set O(εεn) variables at n) variables at random to select just 1 solution from ballrandom to select just 1 solution from ball

The above isolates with subexponential The above isolates with subexponential success probabilitysuccess probability

TheoremTheorem sskk ≤ ≤ σσkk+o(1)+o(1) ss∞∞ = = σσ∞∞

9/23/04 53

OutlineOutline MotivationMotivation AlgorithmsAlgorithms DesignDesign

SparsificationSparsification IsolationIsolation

ConclusionConclusion

9/23/04 54

OutlineOutline MotivationMotivation AlgorithmsAlgorithms DesignDesign ConclusionConclusion

Fate of k-SATFate of k-SAT Open problemsOpen problems

9/23/04 55

Fate of k-SATFate of k-SAT Last few improvements to best 3-SAT Last few improvements to best 3-SAT

algorithms have improved constant in algorithms have improved constant in exponent by only a puny amount exponent by only a puny amount – .4150, .4117, .4116, .4111 – are these – .4150, .4117, .4116, .4111 – are these techniques running out of steam?techniques running out of steam?

Paradigm shifts yield greater Paradigm shifts yield greater improvements – .667, .415improvements – .667, .415

Conjecture – ETH false (sConjecture – ETH false (s33 = 0). = 0).

9/23/04 56

Open problemsOpen problems sskk = = σσkk?? ETH => sETH => skk increases infinitely often. increases infinitely often.

Does sDoes skk strictlystrictly increase? increase? Local search variants usually involve Local search variants usually involve

clever initial assignments. Can we also clever initial assignments. Can we also find better search directions?find better search directions?

Can resolve SAT analysis be tightened?Can resolve SAT analysis be tightened? Other algorithmic paradigms?Other algorithmic paradigms?

9/23/04 57

ThanksThanks M. Paturi, R. ImpagliazzoM. Paturi, R. Impagliazzo Theory lab denizensTheory lab denizens

upper bounds on k -sat

Documents

np problems

time tinput x1

clauses x1

time x

npc problems

good upper bounds

literals x1

time o2n