Problems and Their Classes

• Solved many problems using different algorithms• For each problem - algorithm and computing time • Problem classes based on computing time of best

known algorithmP classNP Class

P Class

• This group consist of all problem whose computing time are polynomial time

• Computing time is bounded by polynomials of small degree Insertion sort O(n2)Merge and Quick sort O(n logn) Binary search O(log n)Linear ClassQuadratic and Cubic Class

• Also called as tractable (easy) problems

NP Class

• Computing of best known algorithm are non polynomial

• TSP using dynamic programming O(n2 2n), Knapsack O(2 n/2)

• No one was has been able to develop a polynomial time algorithms for this class problems

• Requires vast amount of time to execute even moderate size problems

• Problems that can be solved using super polynomial, exponential time algorithm are called intractable (Hard )

NPC Theory

• Does not provide a method of obtaining polynomial time algorithm for problems of NP Class

• Nor it say that algorithms of polynomial complexity do not exist

• But many of the problems for which there are no known polynomial time algorithms are computationally related

Further Classes of Problems

• NP Complete (Non deterministic polynomial time complete) problems – NPC Problem can be solved in polynomial time if and only if all other NPC problems can also be solved in polynomial time

• NP Hard problems – if NP hard problem can be solved in polynomial time , then all NPC problems can be solved in polynomial time

• No NPC or NPHard problem is polynomially solvable • All NP Complete problems are NP hard but some NP

Hard are not NPC

• There are many problems of NPC and NPH , focus should be on nondeterministic computations (to be defined)

• Relationship of these classes to nondeterministic computations together with apparent power of nondeterminism leads conclusion no NPC or NPH is polynomially solvable

Deterministic and Non Deterministic Algorithms

• Deterministic AlgorithmResult of every operation is uniquely defined Agree with way programs are executed on a computer

• In a theoretical framework we can remove this restriction on the outcome of every operation

• Non Deterministic AlgorithmAllow algorithm to contain operations whose outcomes

are limited to specific set of possibilities Machine executing such operations are allowed to

choose any one of these outcome subject to termination condition defined

This leads to non deterministic Algorithm

Non Deterministic Algorithms

• Introduce new functionsChoice (set S) – arbitrarily choose one element of SFailure() – signals an unsuccessful completionSuccess()- signals a successful completion X = choice(1,n) results in x being assigned any one of

integer in the range of 1 to n, but no rule how this choice is to be made !

• Failure and success – just to define a computation of algorithm and cannot be used to effect a return

• Whenever there is a set of choices that leads to a successful completion, then one such set of choices is always made and algorithm terminates successfully

Non Deterministic Algorithms

• NDA terminates unsuccessfully if and only if there is no set of choices leading to success signal

• Hence depending upon order in which choices are made – affects the successful / unsuccessful termination of NDA

• Computing time of Choice(), Success(),failure() is O(1) – Constant

• Machine capable of handling NDA is called as nondeterministic Machine (NDM)

• Although, NDM do not exist in practice, there are certain problems that can not be solved by deterministic algorithm.

Non Deterministic Search Algorithms

• A decision problem gives answer either 0 or 1• Search a element x in a given set A, write index else 0

J = choice (1,n); // Select form a set If A[j] = x then {write (j);success();} //write indexWrite(0); failure();

• Computing time of this algorithm is O(1) in both cases , HOW ?

• Any Deterministic search algorithm requires O(n) time – Linear

Non Deterministic Sorting

1. Algorithm Nsort(A,N)

2. // sort n positive numbers

3. {

4. For I = 1 to n do B[i] = 0 ; // initialize B

5. For I = 1 to n do

6. {

7. J = choice(1,n);

8. If B[j] != 0 then failure();

9. B[j] = A[i];

10. }

11. For I = 1 to n-1 do // verify the order

12. If B[i] > B[i+1] then failure ();

13. Write (B);

14. Success();

15. }

• In the for loop of 5 to 10 each A[i] is assigned to position in B

• Line 7 non deterministically identifies this position

• Line 8 checks whether this position is already used or not !

• The order of numbers in B is some permutation of the initial order in A

• For loop of line no 11 and 12 verifies the B is in ascending order,

• Since there is always a set of choices at line no 7 for ascending order, this is a sorting algorithm of Complexity O(N)

• All deterministic sorting algorithms must have A complexity O(nlogn)

Non Deterministic Algorithms

• A deterministic interpretation of a non deterministic algorithm can be made by allowing unbounded parallelism in computation

• Each time a choice is to be made, the algorithm makes several copies of itself

• One copy for each possible choices, many copies are executing at the same time.

• First copy to reach successful completion terminates all other copies

• If a copy reaches to failure completion then only that copy of algorithm terminates

• This interpretation enable us to better understand NDA

Non Deterministic Machine

• NDM does not makes any copies of an algorithm every time choice is made

• NDM has ability to select a correct element from set of choices (if exists)

• A correct element is defined relative to a shortest sequence of choices that leads to successful completion

• If no sequence of choices leading to successful termination then unsuccessful termination / computation with one unit of time

• Machine is Fictitious, no concerns with how the machine can make a correct choice at each step

Definitions

• Decision problem - a problem having the answer either 0 or 1. Decision problem can be NPC

• Decision Algorithm – solves a decision problem. Gives output 1 on successful completion and 0 on unsuccessful termination

• Optimization Problem – problem involving the identification of an optimal value (either maximum or minimum) for a given cost function. Optimization problem may be NP Hard

• Satisfiability problem – find out whether a given expression is true for some assignment of truth values to the variable in that expression

Definitions

• Clique – maximal complete subgraph for a given graph, size of Clique is the number of vertices in it

• Reducibility – Let L1 and L2 are 2 problems. L2 can be solved in polynomial time using a deterministic algorithm. If the same algorithm can solve problem L1 in deterministic polynomial time, then L1 α L2 (L1 Reduces to L2)

• α is a transitive relation

NDA

• It is possible to construct NDA for which many different choices lead to completions.

• Ex. If numbers of Array A are equal , many different permutations will result in sorted sequence.

• Concern with NDA that generate unique output• Successful completion if output 1• Output 0 if no sequence of choices leading to a

successful completion.• Output statements are implicit in signals success and

failure

• Idea of decision algorithm may appear very restrictive but, many optimization problems can reformed into decision problems with propertyDecision problem can be solved in polynomial time if

and only if the corresponding optimization problem can

• In other cases, we can at least make statement that if the decision problem cannot be solved in polynomial time then the optimization problem cannot either.

Maximal Clique

• A maximal complete subgraph of a graph G(V,E) is clique.

• Size of clique is the number of vertices in it. • Max Clique problem is an optimization problem that

has to determine the size of largest Clique in G• Corresponding decision problem is to determine

whether G has a clique of size at least k for some give k.

Maximal Clique – Decision Problem

• Let Dclique(G,k) be a deterministic decision algorithm for the Clique decision problem

• Size of max clique can be found by different values of k• K = n, k= n-1, k= n-2 ,…. Until Dclique output = 1• If time complexity of Dclique = f(n) then size of max

clique can be found in time <=n.f(n)• Also if size of max clique can be determined in time

g(n), then the decision problem can be solved in time g(n)

• Max clique problem can be solved in polynomial if and only if the clique decision problem can be solved in polynomial time

Non Deterministic Clique Pseudocode

1. Algorithm DCK(G,n,k)

2. {

3. S=0; // s is initially empty set

4. For i= 1to k do

5. {

6. T = choice (1,n);

7. If t € S then failure ();

8. S = s union {t} // add t to S

9. }

10. // at this point S contains k distinct vertex indices

11. For all pairs(I,j) such that i€S, j€S and i!=j do

12. If (I,j) is not an edge of G then failure();

13. Success()

14. }

• Algorithm begins by trying to form a set of k distinct vertices

• Tests to see whether these vertices form a complete subgraph.

Satisfiability problem

• Let x1,x2 variables and x1’,x2’ are negation • Literal is either a variable or its negation• Disjunctive normal form (CNF) – (x1^ x2)V(x3^x4’)

AKA Sum of Product • Conjunctive normal form (CNF) –

(x3Vx4’)^(x1VX2’) - AKA Product of Sum• Satisfiability problem is to determine whether a

formula is true for some assignment of truth values to the variables.

Satisfiability problem

1. Algorithm eval (E,n)

2. // Determine whether the propositional formula E is satisfiable

3. For i=1 to n do // choose a truth value assignment

4. X = choice(false,true);

5. If E(x1,x2,…xn)then success();

6. Else failure();

7. }

• Algorithm terminates successfully if and only if a given propositional formula E(x1,x2,….xn) is satisfiable

• Algorithm proceed by simply choosing non deterministically one of the 2 n possible assignements of truth values to x1,x2,…xn and verifying that E is true for that assignment

• Time taken is O(n) to choose values of x1,x2,….xn plus the time needed to deterministically evaluate E for that assignment which is proportional to E

Relation between p, np, npc and nph• Deterministic Algorithm A

and input I, halting problem is to determine that whether A will stop or enter in an infinite loop.

• Deterministic algorithms are special case of non deterministic algorithms • What we do not know and

what has become the most famous unsolved problem in computer science is whether P = NP or P != NP

p

np

npc

nph

Cook theorem

• Is there any single problem in NP such that if we showed it to be in P, then that would imply that P = NP?

• Satisfiability is in P if and only if P = NP

NP Hard and NPC

• A problem L is NP Hard if and only if Satisfiability reduces to L (Satisfiability α L)

• A problem is NPC if and only if L is NP hard and L € NP

• Only a decision problem can be NPC• Optimization problems may be NPH• If L1 is decision problem and L2 is optimization

problem is possible L1 α L2 • Ex. Knapsack decision problem reduces to knapsack

optimization problem• Clique decision problem reduces to Clique

optimization problem

NP Hard and NPC

• Optimization problems may reduces to corresponding decision problem

• Optimization problems cannot be NP Complete where as decision problem can

• There also exist NP Hard decision problems that are not NPC

Halting problem of Deterministic algorithm

• Arbitrary deterministic algorithm A with input I ever terminates (or enters into infinite loop)

• This is well known undecidable problem, No algorithm (of any complexity), Clearly cannot be in NP

• To show Satisfiability α halting problem, construct algorithm A, whose input are propositional to formula X.

• X has n variables and A tries 2n possible truth assignments and verifies X is satisfiable

• A stops if X is satisfiable or enters in an infinite loop

Halting problem of Deterministic algorithm

• If we had a polynomial time algorithm for halting problem then we could solve Satisfiability problem in polynomial time using Algorithm A and input X

• Halting problem is NP HARD NOT NP

Polynomially equivalence

• Two problems L1 and L2 are said to be polynomially equivalent if and only if L1 α L2 and L2 α L1

• Let L1 is NP Hard, To show L2 as NP Hard, show L1 α L2

• We know α is transitive relation, it follows if Satisfiability α L1 and L1 α L2 then Satisfiability α L2

• To show that an NP HARD decision problem is NPC exhibit a polynomial time nondeterministic algorithm for it

CNF Satisfiability α Clique Decision Problem (CDP)

• Let F = ᴧ 1<=i<=k Ci be a propositional formula in Conjunctive normal form (CNF)

• Let 1<= I <=n be the variables in F• Construct a graph G=(V,E) from F, such that G has a

clique of size at least k if and only if F is satisfiable • If we have a polynomial time algorithm for CDP then

we can obtain a polynomial time algorithm for CNF • For any F, Define graph G as V = {(σ,i)|σ is literal in

clause Ci} E= {(σ,i),(δ,j)|i≠j and σ≠δ’}

Example

• F=(x1 v x2 v x3)ᴧ(x1’v x2’v x3’)

(x1,1) (x1’,2)

(x2,1) (x2’,2)

(x3,1) (x3’,2)

• By setting x1 = true and x2’ = true F is satisfied

Node Cover decision problem (NCDP)

• A set S € V is a node cover for a graph G(V,E) if and only if all edges in E are incident to at least one vertex in S.

• Size of |S| of the cover is the number of vertices in S• 1 2 S = {2,4} is anode cover of size 2• 3 S = {1,3,5} size 3• 5 4• In NCDP , we are given a graph and integer K,

determine whether G has a node cover of size K

CDP α NCDP

• Let G (V,E) and K define in instance in CDP, |V|=n• Construct G’ (complement of a graph) with a node

cover of size at most n-k if and only if G has a clique of size at least k

• G G’• G’ has node cover of {4,5}• G has Clique of size 5-2 = 3 with node 1-2-3

1

2

3

5

4

1

2

3

5

4

Polynomial Reductions

• Problem P is reducible to QP p QTransforming inputs of P

to inputs of Q

• Reducibility relation is transitive.

Dynamic Programming

Algorithmic Analysis

Dynamic Programming

• Dynamic programming is typically applied to optimization problems. In such problem there can be many solutions. Each solution has a value, and we wish to find a solution with the optimal value.

Divide-and-conquer

• Divide-and-conquer method for algorithm design:

• Divide: If the input size is too large to deal with in a straightforward manner, divide the problem into two or more disjoint subproblems

• Conquer: conquer recursively to solve the subproblems

• Combine: Take the solutions to the subproblems and “merge” these solutions into a solution for the original problem

Dynamic programming

• Dynamic programming is a way of improving on inefficient divide-and-conquer algorithms.

• By “inefficient”, we mean that the same recursive call is made over and over.

• If same subproblem is solved several times, we can use table to store result of a subproblem the first time it is computed and thus never have to recompute it again.

• Dynamic programming is applicable when the subproblems are dependent, that is, when subproblems share subsubproblems.

Ex1:Assembly-line scheduling

• Automobiles factory with two assembly lines.

Each line has the same number “n” of stations. Numbered j = 1, 2, ..., n.

We denote the jth station on line i (where i is 1 or 2) by Si,j .

The jth station on line 1 (S1,j) performs the same function as the jth station on line 2 (S2,j ).

The time required at each station varies, even between stations at the same position on the two different lines, as each assembly line has different technology.

time required at station Si,j is (ai,j) .

There is also an entry time (ei) for the chassis to enter assembly line i and an exit time (xi) for the completed auto to exit assembly line i.

Ex1:Assembly-line scheduling

• After going through station Si,j, can either

Job stay on same line next station is Si,j+1

no transfer cost , or

job transfer to other line next station is Si+1,j+1

transfer cost from Si,j to Si+1,j+1 is ti,j ( j = 1, … , n–1)

40

Ex1:Assembly-line scheduling

•(Time between adjacent stations are nearly 0).

Optimal Solution Structure

• optimal substructure : choosing the best path to Sij.

• The structure of the fastest way through the factory (from the starting point)

• The fastest possible way to get through Si,1 (i = 1, 2)

Only one way: from entry starting point to Si,1

take time is entry time (ei)

Step 1: Optimal Solution Structure

• The fastest possible way to get through Si,j (i = 1, 2) (j = 2, 3, ..., n). Two choices:

Stay in the same line: Si,j-1 Si,j

Time is Ti,j-1 + ai,j

If the fastest way through Si,j is through Si,j-1, it must have taken a fastest way through Si,j-1

Transfer to other line: Si+1,j-1 Si,j

Time is Ti+1,j-1 + t3-i,j-1 + ai,j

Optimal Solution Structure

• An optimal solution to a problem finding the fastest way to get through Si,j

• contains within it an optimal solution to sub-problems finding the fastest way to get through either Si,j-1 or Si+1,j-1

•

Example

Recursive Solution

• Define the value of an optimal solution recursively in terms of the optimal solution to sub-problems

• Sub-problem here finding the fastest way through station j on both lines (i=1,2) Let fi [j] be the fastest possible time to go from starting point through Si,j

• The fastest time to go all the way through the factory: f*

• x1 and x2 are the exit times from lines 1 and 2, respectively

Thank you