introduction to algorithms: brute-force...
TRANSCRIPT
![Page 1: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/1.jpg)
Introduction to Algorithms:Brute-Force Algorithms
![Page 2: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/2.jpg)
Introduction to Algorithms
Brute Force• Powering a Number• Selection Sort• Exhaustive Search
• 0/1 Knapsack Problem• Assignment Problem
CS 421 - Analysis of Algorithms2
![Page 3: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/3.jpg)
Brute-Force Algorithm Design
CS 421 - Analysis of Algorithms3
• Straightforward, usually based on problem definition.
• Rarely the most efficient but can be applied to wide range of problems.
• For some elementary problems, almost as good as most efficient. May not be worth cost.
• May work just as well on smaller data sets.• Used to measure other algorithms against.
![Page 4: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/4.jpg)
Calculating Powers of a Number
CS 421 - Analysis of Algorithms
Problem: Compute a n, where n ∈N.
Naive algorithm: Θ(n).
a n = n * n * n * … * n
4
a times
![Page 5: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/5.jpg)
Selection Sort
CS 421 - Analysis of Algorithms
Given a list of n orderable items, rearrange them in non-decreasing order.1. Scan entire list to find smallest item.2. Exchange it with first item. First element now in its
final, sorted position.3. Scan remaining n – 1 items, starting with second
element, and find smallest item.4. Exchange it with second item. Second element now in
its final, sorted position. 5. Repeat for a total of n – 1 times.
5
![Page 6: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/6.jpg)
Selection Sort -Example
CS 421 - Analysis of Algorithms
Selection sort on the list: 89, 45, 68, 90, 29, 34, 17. Each line corresponds to an iteration of the algorithm. The values in bold are the smallest item for that iterations. Elements to the left of the vertical bar are in their final positions.
6
| 89 45 68 90 29 34 1717 |45 68 90 29 34 8917 29 | 68 90 45 34 8917 29 34 | 90 45 68 8917 29 34 45 | 90 68 8917 29 34 45 68 | 90 8917 29 34 45 68 89 | 90
![Page 7: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/7.jpg)
Selection Sort -Analysis
CS 421 - Analysis of Algorithms
Selection sort is implemented using nested forloops:
Outer loop: iterates from 0 to n – 2 – don’t have to visit element because already in final sorted position
Inner loop: finds smallest value remaining the list. Even though gets smaller each iteration, still on order of n.Therefore:
T(n) = Θ(n2)7
![Page 8: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/8.jpg)
Sequential search
In a list No better than linear timeWe can do better if not a listWorst case O(n)
CS 421 - Analysis of Algorithms 8
![Page 9: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/9.jpg)
Exhaustive Search
CS 421 - Analysis of Algorithms9
Brute-force approach to combinatorial problems (i.e. permutations, combinations, subsets of a given set).1. Generate all elements of problem domain (all
possible solutions).2. Select those that satisfy the constraints of the
problem.3. Chose one or more that are most desirable (i.e.
optimize the objective function).
![Page 10: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/10.jpg)
0/1 Knapsack ProblemAnalysis
CS 421 - Analysis of Algorithms10
The most costly operation is generating all of the subsets of n items. Since there are 2n subsets,
Brute-force Approach to Knapsack problem: Ω(2n).
Not feasible for any but the smallest values of n.
![Page 11: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/11.jpg)
0-1 knapsack Algorithmpublic boolean knapsack (double [] v, double V, double [] w, double W,
double counterV, double counterW, int i){if(i<v.length){
boolean b1=knapsack(v,V,w,W, counterV,counterW, i+1);boolean b2=knapsack(v,V,w,W,counterV+v[i],counterW+w[i], i+1);return b1||b2;
}else{return (counterV>= V) && (counterW<= W)
}}
T(n)=2T(n-1) +1T(n-1)<=C*2^{n-1}T(n)<=C*2*2^(n-1)+1T(n)<=C*2*2^(n-1)+1<=C*2^n
![Page 12: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/12.jpg)
0-1 knapsack Algorithmpublic boolean knapsack (double [] v, double V, double [] w, double W,
double counterV, double counterW, int i){if(i<v.length){
boolean b1=knapsack(v,V,w,W, counterV,counterW, i+1);boolean b2=knapsack(v,V,w,W,counterV+v[i],counterW+w[i], i+1);return b1||b2;
}else{return (counterV>= V) && (counterW<= W)
}}
T(n)=2T(n-1) +1T(n-1)<=C*2^{n-1}T(n)<=C*2*2^(n-1)+1T(n)<=C*2*2^(n-1)+1<=C*2^n
T(n)=2T(n-1) +1T(n-1)<=C*(n-1)2^{n-1}T(n)<=C*2*2^(n-1)*(n-1)+1T(n)<=C*2*2^(n-1)*(n-1)+1<=C*n*2^n
![Page 13: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/13.jpg)
0/1 Knapsack Problem
CS 421 - Analysis of Algorithms13
Given n items of known weights w1, w2, …, wn and values v1, v2, …, vn , and a knapsack of capacity W, find most valuable subset of items that will fit.
1. Generate all subsets of n items.2. Calculate the weights of each and eliminate all
the infeasible solutions.3. Find the subset with the maximum value.
![Page 14: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/14.jpg)
Assignment Problem
CS 421 - Analysis of Algorithms 14
There are n jobs that need to be completed, and n people that need to be assigned to a job, one person per job. The cost for the i th person to perform job j is known C[i, j]. Find the assignment with the lowest cost.
1. Generate all permutations of n people assigned to n jobs.
2. Calculate the cost of each permutation/solution.3. Find the solution with the minimum value.
![Page 15: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/15.jpg)
Assignment ProblemAnalysis
CS 421 - Analysis of Algorithms15
The most costly operation is generating all of the permutations. Since there are n! permutations,
Brute-force Approach to Assignment problem: Θ(n!).
Not feasible for any but the smallest values of n.
![Page 16: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/16.jpg)
String Matching
CS 421 - Analysis of Algorithms 16
![Page 17: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/17.jpg)
String Matching
CS 421 - Analysis of Algorithms 17
![Page 18: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/18.jpg)
String Matching
CS 421 - Analysis of Algorithms 18
![Page 19: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/19.jpg)
Complexity
m(n-m+1) O(nm)
CS 421 - Analysis of Algorithms 19
![Page 20: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/20.jpg)
Travelling salesman problem
CS 421 - Analysis of Algorithms 20
![Page 21: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/21.jpg)
Travelling salesman problem
CS 421 - Analysis of Algorithms 21
Path={(A,B),(B,C),(C,D),(D,A)}
![Page 22: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/22.jpg)
Travelling salesman problem
CS 421 - Analysis of Algorithms 22
Path={(A,B),(B,C),(C,D),(D,A)}Path={(A,B),(B,D),(D,A),(A,B), (B,A)}
![Page 23: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/23.jpg)
Hamiltonian Path
CS 421 - Analysis of Algorithms 23
Path={(A,B),(B,C),(C,D),(D,A)}Path={(A,B),(B,D),(D,A),(A,B), (B,A)}
![Page 24: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/24.jpg)
Hamiltonian Path
CS 421 - Analysis of Algorithms 24
Path={(A,B),(B,C),(C,D),(D,A)}Path={(A,B),(B,D),(D,A),(A,B), (B,A)}
A Hamiltonian path or traceable path is a path that visits each vertex exactly once
![Page 25: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/25.jpg)
Hamiltonian Path
CS 421 - Analysis of Algorithms 25
Path={(A,B),(B,C),(C,D),(D,A)}Path={(A,B),(B,D),(D,A),(A,B), (B,A)}
A Hamiltonian path or traceable path is a path that visits each vertex exactly once
![Page 26: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/26.jpg)
Hamiltonian Path
CS 421 - Analysis of Algorithms 26
Path*={(A,B),(B,C),(C,D),(D,A)}Path={(A,B),(B,D),(D,A),(A,B), (B,A)}
A Hamiltonian path or traceable path is a path that visits each vertex exactly once
Optimal
![Page 27: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/27.jpg)
Graph Coloring
CS 421 - Analysis of Algorithms 27
![Page 28: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/28.jpg)
Binary counter
CS 421 - Introduction to Algorithms 28
When increment up to n, change is at most 1+floor(log n) bits -number of bits in the binary representation of n.
Thus, the cost of counting up to n, a sequence of nincrements, is O(n log n).
Counter value
A[k] A[k-1] … A[2] A[1] A[0] Cost
0 0 0 … 0 0 0 0
1 0 0 … 0 0 1 1
2 0 0 … 0 1 0 2
3 0 0 … 0 1 1 1
4 0 0 … 1 0 0 3
5 0 0 … 1 0 1 1
6 0 0 … 1 1 0 2
7 0 0 … 1 1 1 1
8 0 0 … 0 0 0 4
… … … … … … …
2k - 1 1 1 … 1 1 1 k
![Page 29: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/29.jpg)
Binary counter
CS 421 - Analysis of Algorithms 29
454 Chapter 17 Amortized Analysis
analysis, we assign the amortized cost of each operation to be the average cost. Inthis example, therefore, all three stack operations have an amortized cost of O.1/.
We emphasize again that although we have just shown that the average cost, andhence the running time, of a stack operation is O.1/, we did not use probabilisticreasoning. We actually showed a worst-case bound of O.n/ on a sequence of noperations. Dividing this total cost by n yielded the average cost per operation, orthe amortized cost.
Incrementing a binary counterAs another example of aggregate analysis, consider the problem of implementinga k-bit binary counter that counts upward from 0. We use an array AŒ0 : : k ! 1! ofbits, where A: length D k, as the counter. A binary number x that is stored in thecounter has its lowest-order bit in AŒ0! and its highest-order bit in AŒk ! 1!, so thatx D
Pk!1iD0 AŒi ! " 2i . Initially, x D 0, and thus AŒi ! D 0 for i D 0; 1; : : : ; k ! 1. To
add 1 (modulo 2k) to the value in the counter, we use the following procedure.
INCREMENT.A/
1 i D 02 while i < A: length and AŒi ! == 13 AŒi ! D 04 i D i C 15 if i < A: length6 AŒi ! D 1
Figure 17.2 shows what happens to a binary counter as we increment it 16 times,starting with the initial value 0 and ending with the value 16. At the start ofeach iteration of the while loop in lines 2–4, we wish to add a 1 into position i .If AŒi ! D 1, then adding 1 flips the bit to 0 in position i and yields a carry of 1,to be added into position i C 1 on the next iteration of the loop. Otherwise, theloop ends, and then, if i < k, we know that AŒi ! D 0, so that line 6 adds a 1 intoposition i , flipping the 0 to a 1. The cost of each INCREMENT operation is linearin the number of bits flipped.
As with the stack example, a cursory analysis yields a bound that is correct butnot tight. A single execution of INCREMENT takes time ‚.k/ in the worst case, inwhich array A contains all 1s. Thus, a sequence of n INCREMENT operations onan initially zero counter takes time O.nk/ in the worst case.
We can tighten our analysis to yield a worst-case cost of O.n/ for a sequence of nINCREMENT operations by observing that not all bits flip each time INCREMENTis called. As Figure 17.2 shows, AŒ0! does flip each time INCREMENT is called.The next bit up, AŒ1!, flips only every other time: a sequence of n INCREMENT
![Page 30: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/30.jpg)
NP-Hard
NP-HardOptimized approach:n Cut n Branch and Bound
CS 421 - Analysis of Algorithms 30
![Page 31: Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at](https://reader035.vdocuments.mx/reader035/viewer/2022081621/6125d09a3e9dc214c658e3d6/html5/thumbnails/31.jpg)
Conclusion
CS 421 - Analysis of Algorithms
The Brute Force approach:• Straightforward approach to solving
problem, usually based directly on problem description.
• Wide applicability and simple.• But in general, has subpar performance.
31