1

Design and Analysis of Algorithms

Yoram MosesLecture 11

June 3, 2010

http://www.ee.technion.ac.il/courses/046002

2

Nondeterministic Polynomial Time(NP)

3

Shortest Path: Search, Existence, Verification

Search problem: Input: (G,w,s,t): a directed graph G with weight function w,

a source s, and a sink t. Goal: find a shortest path from s to t. (or reject if none exists) Complexity: our solution runs in O(VE) = O(n2)

(Notice n = size of input = O(V+E)) Existence problem:

Input: (G,w,s,t,k): G,w,s,t are as before + a number k. Goal: decide whether there is a path from s to t of length ≤ k. Complexity: our solution runs in O(VE) = O(n2)

Verification problem: Input: (G,w,s,t,k,p): G,w,s,t,k as before. p is a path in G. Goal: decide whether p is a simple path from s to t of length ≤ k. Complexity: O(V) = O(n).

4

Max Flow: Search, Existence, Verification

Search problem: Input: (G,c,s,t): a directed graph G with capacity function c, a

source s, and a sink t. Goal: find a maximum flow in G. (or reject if none exists) Complexity: O(VE2) = O(n3)

Existence problem: Input: (G,c,s,t,k): G,c,s,t as before + a number k. Goal: decide whether there is a flow in G with value ≥ k. Complexity: O(VE2) = O(n3)

Verification problem: Input: (G,c,s,t,k,f): G,c,s,t,k as before. f is a function from edges of

G to real numbers. Goal: decide whether f is a legal flow with value ≥ k. Complexity: O(E) = O(n).

5

Hamiltonian Cycle: Search, Existence, Verification

Search problem: Input: an undirected graph G. Goal: find a Hamiltonian cycle in G (or reject if none exists). Complexity: O(VxV!) = O(n2n log n)

Existence problem: Input: an undirected graph G. Goal: decide whether G has a Hamiltonian cycle. Complexity: O(VxV!) = O(n2n log n)

Verification problem: Input: (G,p): an undirected graph G and a sequence of nodes p. Goal: decide whether p is a Hamiltonian cycle in G. Complexity: O(V) = O(n).

6

3-Coloring: Search, Existence, Verification

Search problem: Input: G: an undirected graph Goal: find a 3-Coloring of G. (or reject if none exists) Complexity: O(E 3V) = O(n2n log 3)

Existence problem: Input: G: as before. Goal: decide whether G has a 3-Coloring. Complexity: O(E 3V) = O(n2n log 3)

Verification problem: Input: (G,): G as before and : V {1,2,3}. Goal: decide whether is a 3-Coloring of G. Complexity: O(E) = O(n).

7

Search and Existence vs. Verification

Conclusion: in many natural examples:Search and existence are computationally

equivalentVerification is easier

Sometimes it’s just a little easier (Shortest Path, Max flow)

Sometimes it’s a lot easier (Hamiltonian cycle, 3-Coloring)

8

Verification Relations Language: L {0,1}*

Definition: A verification relation for L is a relation R {0,1}* {0,1}* s.t. for all x {0,1}*: x L there is at least one y {0,1}* s.t. (x,y) R. x L there is no y {0,1}* s.t. (x,y) R.

y is called the “certificate” for x A.k.a. its “witness” or “proof”

Remarks: Every input x L has at least one certificate y. If (x,y) R, then y is a certificate for x. An input x L may have several certificates. A language L has many verification relations.

9

Verification Relations: Examples Shortest path:

x = (G,w,s,t,k), y = a path p Language: {(G,w,s,t,k): G has an s-t path of length ≤ k} Certificate: s-t path of length ≤ k Verification relation:

{((G,w,s,t,k),p): p is an s-t path of length ≤ k in G}

Hamiltonian cycle: x = undirected graph G, y = a path p Language: G that has a Hamiltonian cycle Certificate: a Hamiltonian cycle in G Verification relation: {(G,p): p is a Hamiltonian cycle in G}

10

Nondeterministic Polynomial Time

Definition: A binary relation R is polynomially bounded, if there exists some c > 0 s.t. for every (x,y) R, |y| ≤ |x|c.

Definition: L is polynomial-time verifiable, if it has a verification relation R, which satisfies both: R is polynomially bounded, and R is polynomial-time decidable.

Definition: The class NP (Nondeterministic Polynomial Time) is the set of all polynomial-time verifiable languages.

11

NP: Examples

Examples of languages in NP: Decision Shortest Path, Decision Max Flow, Decision LP Hamiltonian Cycle, TSP, 3-Coloring, SAT, k-SAT, Clique

Examples of languages not known to be in NP: HC-complement: given a graph G, decide whether G has no

Hamiltonian cycles.

12

Definition: A nondeterministic algorithm is an algorithm N that, on input x, First, N “nondeterministically” guesses a “witness” y. Then, N runs a deterministic “verification” algorithm on (x,y). Note: N may make different nondeterministic guesses in different

runs on the same input x.

Nondeterministic Algorithms

Nondeterministic guess

Nondeterministic Algorithm N

Verificationxy

yes/no(x,y)

13

Decision by Nondeterministic Algorithms

Definition: A nondeterministic algorithm N is said to decide language L if: For every input x L, there is at least one guess y s.t. N accepts (x,y). For every input x L, the verification algorithm N rejects (x,y), for all

guesses y.

A polynomial-time nondeterministic algorithm is one in which The guesses (y’s) are of polynomial size (in |x|), and The verification algorithm runs in polynomial time.

Lemma: L NP iff L is decidable by a polynomial-time nondeterministic algorithm.

14

An NP Algorithm for Clique

Nondeterministic guess (input: x = (G,k))1. for i = 1,…,k 2. vi nondeterministic guess of a node in V=V(G)3. output y = (v1,…,vk)

Verification algorithm (input: (x,y))1. If x is not a valid encoding of a graph G and an integer k, reject.2. If y is not a valid encoding of k nodes v1,…,vk in G, reject.3. If v1,…,vk are not distinct, reject.4. for i 1,…,k-1 do5. for j i+1,…,k do6. if {vi,vj} E reject.7. accept

15

Lemma: P NP Biggest open problem of computer science:

is P = NP?

Two possibilities:

Current belief: P NP Search & Existence strictly harder than Verification.

P = NP?

P vs. NP

P = NPP

NP

16

f: N N: a complexity measure. Time(f(n)) = all languages decidable in time O(f(n)).

Lemma: Let f(n),g(n) be two complexity measures. If there exists a constant c, s.t. for all n > c, f(n) ≤ g(n), then

Time(f(n)) Time(g(n)).

Theorem (Time Hierarchy)Let f(n),g(n) be two complexity measures. If there exists a constant c, s.t. for all n > c, f(n) ≤ g(n)1/2, then

Time(f(n)) Time(g(n)).

Time Hierarchy

17

P

Definition:

Lemma: P EXP but P EXPLemma: NP EXP (exercise)Open problem: is NP = EXP? 3 Possibilities:

P, NP, and EXP

P

EXP

NP

P

EXP = NP

NP =P

EXP

18

NP-Completeness (NPC)

Problems in NP not known to be in P:

Hamiltonian Cycle, Clique, SAT, k-SAT (k ≥ 3), k-Coloring (k ≥ 3), TSP, …. (many others)

All of these are “NP-Complete” NP-Complete Problems:

Belong to NP If any of them belongs to P, then NP = P.

Two possibilities:

NPCP

NPNP =

NPC = P

19

NP-Hardness (NPH)

Definition: A language L is NP-hard if L’ ≤p L holds for all L’ NP.

NPH = class of all NP-hard problems.

Lemma: If any NP-hard problem belongs to P, then NP = P. If one NPH problem is easy, then all of NP is easy.

Lemma: If L NPH and L ≤p L’, then L’ NPH.

20

NP-Completeness

Definition: A language L is NP-complete if both L NP and L is NP-hard

NPC = class of NP-complete problems NPC = NP NPH

Theorem: If some NPC language is in P, then P = NP.

(P NPC NP = P = NPC). If some NPC language is not in P, then no NPC language is in P.

(NPC P P NPC = NP P).

21

NP-Completeness

NPC: “hardest” problems in NP Behave as a “single block”: either all in P

or all outside P

Lemma: If L1,L2 NPC, then both

L1 ≤p L2 and L2 ≤p L1.

22

Proving NP-Completeness

How to prove that a given language L is NPC? Show that L NP, and Show that L’ ≤p L holds for every L’ NP.

Easier alternative: Show that L NP, and Find some NPC problem L’ and show L’ ≤p L.

How do we obtain the first NPC problem? Using the first alternative Cook-Levin theorem: Circuit-SAT is NP-complete.

23

NP-Completeness: the Full Recipe

To show that L is NPC:Prove L is in NP

Show a polynomial time nondeterministic algorithm for L

Select an NPC problem L’Show a polynomial-time reduction f from L’ to L

Prove that x L’ iff f(x) L Show a polynomial-time algorithm to compute f

24

Example: Clique is NPC

Clique is in NP (seen today) 3-SAT is NPC (will show this later on) 3-SAT ≤p Clique (seen in previous lecture) Therefore: Clique is also NP-Complete!

25

End of Lecture 11