theory of computation chapter 1:...
TRANSCRIPT
![Page 1: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/1.jpg)
Theory of Computation
Chapter 1: Introduction
Guan-Shieng Huang
Sep. 20, 2006Feb. 9, 2009
0-0
![Page 2: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/2.jpg)
'
&
$
%
Text Book
Computational Complexity, by C. H. Papadimitriou,
Addison-Wesley, 1994.
1
![Page 3: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/3.jpg)
'
&
$
%
References
• Garey, M.R. and D.S. Johnson, Computers and Intractability:
A Guide to the Theory of NP-Completeness. 1979: W. H.
Freeman and Company.
• Ausiello, G., et al., Complexity and Approximation:
Combinatorial Optimization Problems and Their
Approximability Properties. 2003: Springer-Verlag.
• Vazirani, V.V., Approximation Algorithms. 2004: Springer
Verlag.
• Sipser, M., Introduction to the Theory of Computation. 2 ed.
2005: Course Technology.
• Kozen, D.C., Theory of Computation. 2006: Springer Verlag.
• Goldreich, O., Computational Complexity: A Conceptual
Perspective. 2008: Cambridge University Press.
2
![Page 4: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/4.jpg)
'
&
$
%
• Arora, S., Computational Complexity: A Modern Approach.
Draft, 2006.
(http://www.cs.princeton.edu/∼arora/book/book.html)
3
![Page 5: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/5.jpg)
'
&
$
%
Scope
• Turing Machines (Chap. 2)
• Computability (Chap. 3)
• Boolean Logic (Chap. 4)
• Relations Between Complexity Classes (Chap. 7)
• Reductions and Completeness (Chap. 8)
• NP-complete Problems (Chap. 9)
• NP and coNP (Chap. 10)
• Primality Testing (Chap. 10)
4
![Page 6: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/6.jpg)
'
&
$
%
• Randomized Computation (Chap. 11)
• Cryptography (Chap. 12)
• Approximability (Chap. 13)
5
![Page 7: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/7.jpg)
'
&
$
%
Outline of Chap. 1
• Graph Reachability
• Maximum Flow
• Matching
• Traveling Salesman Problem
6
![Page 8: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/8.jpg)
'
&
$
%
Graph Reachability
Problem 1 Given a directed graph G = (V, E), where
V = 1, 2, . . . , n, ask whether there is a path from node 1 to node
n.
1
2 3
4
5
1 → 4 → 3 → 5
1
2 3
4
5
7
![Page 9: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/9.jpg)
'
&
$
%
Algorithm
1. Let S = 1.
2. If S is empty, go to 5; otherwise, remove one node, say t, in S.
3. For each edge (t, u) ∈ E, if u is not marked, mark u and add u
to S.
4. Go to 2.
5. If node n is marked, answer “yes”; otherwise answer “no.”
8
![Page 10: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/10.jpg)
'
&
$
%
Problem 1.4.2
1. Show by induction on i that, if v is the ith node added by the
search algorithm to the set S, then there is a path from node 1
to v.
2. Show by induction on ` that if node v is reachable from node 1
via a path with ` edges, then the search algorithm will add v to
set S.
9
![Page 11: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/11.jpg)
'
&
$
%
Example
1
2 3
4
5
S t
1 1
3, 4 3
5, 4 4
5 5
10
![Page 12: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/12.jpg)
'
&
$
%
Complexity
1. Observe that each node can stay in S at most once.
2. Each edge is used at most once.
3. There are at most n2 edges.
4. The time complexity is at most n2.
5. Space complexity is n.
11
![Page 13: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/13.jpg)
'
&
$
%
Remark
1. How to implement Step 2 in the algorithm?
stack ⇒ DFS, queue ⇒ BFS
2. How to implement Step 3? Random access memory.
3. What is the computational model?
4. Big-O.
12
![Page 14: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/14.jpg)
'
&
$
%
Big-O
Let f and g be functions from N to N . We write f(n) = O(g(n)) if
there are positive integers c and n0 such that, for all n ≥ n0,
f(n) ≤ c · g(n).
1. f(n) = O(g(n)) means intuitively that f grows as g or slower.
2. We write f(n) = Ω(g(n)) if g(n) = O(f(n)).
3. We write f(n) = Θ(g(n)) if f(n) = O(g(n)) and
f(n) = Ω(g(n)).
13
![Page 15: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/15.jpg)
'
&
$
%
Examples
1. n = O(n2)
2. n1.5 = O(n2)
3. If p(n) is a polynomial of degree d, then p(n) = Θ(nd).
4. If c > 1 is a positive integer and p(n) any polynomial, then
p(n) = O(cn).
5. lg n = O(n), or (lg n)k = O(n).
14
![Page 16: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/16.jpg)
'
&
$
%
Determining Big-O
1. f(n) = O(g(n)) if
limn→∞
f(n)
g(n)≤ c
for some constant c.
f(n) = O(g(n)) iff
lim supn→∞
f(n)
g(n)≤ c
for some constant c.
2. If f(n) = O(g(n)) and g(n) = O(h(n)), then f(n) = O(h(n)).
(transitivity)
15
![Page 17: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/17.jpg)
'
&
$
%
Problem 1.4.10
Let f(n) and g(n) be any two of the following functions .
Determine whether (i) f(n) = O(g(n)); (ii) f(n) = Ω(g(n));
f(n) = Θ(g(n)):
(a) n2; (b) n3; (c) n2 log n; (d) 2n; (e) nn; (f) nlog n; (g) 22n
; (h)
22n+1
; (j) n2 if n is odd, 2n otherwise.
It is easy to see that (a) ≺ (c) ≺ (b) ≺ (d) ≺ (e).
Also, (f) ≺ (e), and (g) ≺ (h).
16
![Page 18: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/18.jpg)
'
&
$
%
Polyniomial-Time Algorithm
• polynomial time ⇒ practical, efficient
exponential time ⇒ impractical, inefficient (intractable)
• n80 algorithm v.s. 2n
100 algorithm
• worst case v.s. average case
The exponential worst-case performance of an algorithm may
due to a statistically insignificant fraction of the input.
17
![Page 19: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/19.jpg)
'
&
$
%
Maximum Flow
A network N = (V, E, s, t, c) is a graph (V, E) with two specified
nodes s (the source) and t (the sink) such that
• the source has no incoming edges and the sink has no outgoing
edges;
• for each edge (i, j), we are given a capacity c(i, j), a positive
integer.
18
![Page 20: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/20.jpg)
'
&
$
%
A Network
s t
2
3
1
2
1
4
3
19
![Page 21: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/21.jpg)
'
&
$
%
Flow
1. an assignment of a nonnegative integer value f(i, j) ≤ c(i, j) to
each edge (i, j);
2. for each node, other than s and t, the sum of the fs of the
incoming edges is equal to the sum of the outgoing edges;
( ¬y¶i=¬y¥X)
3. the value of a flow is the sum of the flows in the edges leaving s
(or, equivalently, the sum coming to t).
20
![Page 22: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/22.jpg)
'
&
$
%
Maximum Flow Problem
Given a network, find a flow of the largest possible value.
s t
2/1
3/3
1/1
2/2
1/1
4/3
3/1
21
![Page 23: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/23.jpg)
'
&
$
%
Algorithm
s t
2/1
3/0
1/1
2/0
1/0
4/1
3/0
s t
2/1
3/1
1/1
2/0
1/1
4/1
3/1
s t
2/1
3/3
1/1
2/2
1/1
4/3
3/1
s t
1
3
1
2
1
1
1
2
22
![Page 24: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/24.jpg)
'
&
$
%
Another Example
s
C
C
1
C
C
Require O(n3C) time in the worst case!
23
![Page 25: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/25.jpg)
'
&
$
%
Improvements
The shortest-path heuristic can reduce the time to O(n5).
24
![Page 26: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/26.jpg)
'
&
$
%
Bipartite Matching
A bipartite graph is a triple B = (U, V, E) where U = u1, . . . , un,
V = v1, . . . , vn, and E ⊆ U × V .
25
![Page 27: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/27.jpg)
'
&
$
%
Matching
A matching of a bipartite graph B is a set M ⊆ E s.t.
1. |M | = n;
2. for any two edges (u, v), (u′, v′) ∈ M , u 6= u′ and v 6= v′.
26
![Page 28: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/28.jpg)
'
&
$
%
Algorithm
Reduce MATCHING to MAX FLOW.
27
![Page 29: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/29.jpg)
'
&
$
%
Traveling Salesman Problem
Given n cities 1, 2, . . . , n and a nonnegative integer distance di,j
between any two cities i and j, find the shortest tour∑n
i=1 dπ(i),π(j) where π is a permutation on 1, . . . , n.
28
![Page 30: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/30.jpg)
'
&
$
%
Example
10 5 1 11
8 3 4 5
6 16 4 5
20 2 8 2
29
![Page 31: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/31.jpg)
'
&
$
%
Remark
• TSP is NP-hard (its decision version is in fact NP-complete).
• There is no known polynomial-time algorithm for solving TSP.
• If a problem is NP-complete, most computer scientists believe
that there is no polynomial-time algorithm for it.
30
![Page 32: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/32.jpg)
'
&
$
%
Summary
We have discussed
1. Graph Reachability
2. Big-O notation
3. Maximum Flow
4. Bipartite Matching
5. Traveling Salesman Problem
31
![Page 33: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/33.jpg)
'
&
$
%
• MAX FLOW =⇒ REACHABILITY.
• MATCHING =⇒ MAX FLOW.
They are all polynomial-time solvable.
However, we don’t know whether there exists a polynomial-time
algorithm that can solve TSP.
32
![Page 34: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/34.jpg)
'
&
$
%
Reduction
• Reduction is a classical technique, which transforms an
unknown problem to an existing one.
• It usually implies to transform a harder problem into an easier
one.
• However, in complexity theory, we use it in the perverse way.
• When A reduces to B, we say that B can not be easier that A.
( ¶Vreduce¶VÃø)
33
![Page 35: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/35.jpg)
'
&
$
%
Big-O
Big-O captures the asymptotic behavior for comparison of two
positive functions. However, why we ignore the constant coefficient?
34
![Page 36: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/36.jpg)
'
&
$
%
Problem 1.4.4
(a) A directed graph is acyclic if it has no cycles. Show that any
acyclic graph has a source (a node with no incoming edges).
(b) Show that a graph with n nodes is acyclic if and only if its
nodes can be numbered 1 to n so that all edges go from lower
to higher numbers (use the property in (a) above repeatedly).
(c) Describe a polynomial-time algorithm that decides whether a
graph is acyclic.
35
![Page 37: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/37.jpg)
'
&
$
%
Problem 1.4.5
(a) Show that a graph is bipartite (that is, its nodes can be
partitioned into two sets, not necessarily of equal cardinality,
with edges going only from one to the other) if and only if it
has no odd-length cycles.
(b) Describe a polynomial algorithm for testing whether a graph is
bipartite.
36
![Page 38: Theory of Computation Chapter 1: Introductionstaffweb.ncnu.edu.tw/shieng/theory/972/slides/1_introduction.pdf · Theory of Computation Chapter 1: Introduction Guan-Shieng Huang Sep](https://reader033.vdocuments.mx/reader033/viewer/2022060512/5f2b5b7ff3883c337d757b62/html5/thumbnails/38.jpg)
'
&
$
%
Problem 1.4.9
Show that for any polynomial p(n) and any constant c > 0 there is
an integer n0 such that, for all n ≥ n0, 2cn > p(n). Calculate this
n0 when (a) p(n) = n2 and c = 1; (b) when p(n) = 100n100 and
c = 1100 .
37