foundations of algorithms, fourth edition richard neapolitan, kumarss naimipour chapter 2...
DESCRIPTION
Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer. Divide and Conquer. In this approach a problem is divided into sub-problems and the same algorithm is applied to every subproblem ( often this is done recursively) Examples - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/1.jpg)
Foundations of Algorithms, Fourth EditionRichard Neapolitan, Kumarss Naimipour
Chapter 2Divide-and-Conquer
![Page 2: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/2.jpg)
Divide and Conquer
• In this approach a problem is divided into sub-problems and the same algorithm is applied to every subproblem ( often this is done recursively)
• Examples– Binary Search (review algorithm in book)– Mergesort (review algorithm in book)– Quicksort
![Page 3: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/3.jpg)
Figure 2.1 : The steps down by a human when searching with Binary Search. (Note: x = 18)
![Page 4: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/4.jpg)
Complexity of Binary Search
Since this and many other divide and conquer algorithms are recursive you will recall that we can determine their complexity using recurrence relations.For Binary Search we have T(n) = T(n/2) + 1
=[T(n/4)+1]+1 = T(n/22) + 2=[T(n/8+1]+ 2 = T(n/23)+ 3
… =T(n/2k)+k
![Page 5: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/5.jpg)
What is T(n/2k)+k
We if we let k get larger until n=2k thenwe see that k = log2n. Why?Consequently the relation becomes
T(n) = T(1) + log2n T(n) = log2nSince n/2k is 1 if they are equal and T(1) =1
![Page 6: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/6.jpg)
MergeSort
Recall in this algorithm we divide the array into two equal parts and sort each half prior to merging. The recurrence relation is clearly T(n) = 2T(n/2) + nRecall that Merging is O(n) right?
![Page 7: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/7.jpg)
Figure 2.2: The steps done by a human when sorting with Mergesort.
O(n)
T(n/2)T(n/4)
![Page 8: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/8.jpg)
T(n) = 2T(n/2) + n
T(n) = 2T(n/2) + n = 2[ 2T(n/22) + n/2] + n= 22T(n/22) + 2n = 2[2T(n/23) + n/22] +2n=23T(n/23) + 3n
…=2kT(n/2k) + kn
If n=2k then we haveT(n) = nT(1) + (log2n)n = n+ nlog2n
= O(nlog2n)
![Page 9: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/9.jpg)
QuickSortWorks in situ!
Void quicksort(int low, int high){ int pivot; if (high > low){ partition(low, high, pivot);
quicksort(low, pivot-1); quicksort(pivot+1,high);}
![Page 10: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/10.jpg)
Figure 2.3: The steps done by a human when sorting with Quicksort. The subarrays are enclosed in rectangles whereas the pivot points are free.
![Page 11: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/11.jpg)
PartitionStudy this carefully
void partition (int low, int high, int&pivot){ int I,j, pivotitem; pivotitem = S[low]; // select left item (hmmm) j=low; for (i=low+1; i<=high; i++) There are many ways if (S[i] < pivotitem){ to write this function! j++; All have a complexity swap S[i] and S[j]; of O(n). } pivot= j; swap S[low] and S[pivot];}
![Page 12: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/12.jpg)
Complexity of Quicksort
The complexity of this algorithm depends on how good the pivot value selection is . If the value is always in the middle of array then the best case complexity is T(n) = n + 2T(n/2)Which we already have determined is T(n) = n log2 n
![Page 13: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/13.jpg)
Worst case for Quicksort
This clearly will occur if each pivot value is less than (or greater) all the elements of the array. IE the array is split into 1 and n-1 size pieces. This gives a recurrence relation of T(n) = T(1) + T(n-1) + n-1Time to sort left array right array partition
![Page 14: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/14.jpg)
Worst Case analysisT(n) = T(1) + T(n-1) + n-1 = T(n-1) + n Assume the answer is n(n-1)/2 check it out !n(n-1)/2 = 0 + (n-1)(n-2)/2 + n-1 = (n-1)(n-2)/2 + 2(n-1)/2 =((n-1)(n-2)+ 2(n-1))/2 = (n-1)(n-2+2)/2 = n(n-1)/2 ☺
![Page 15: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/15.jpg)
Quick Sort AnalysisQuicksort’s worst case is θ(n2)Does this mean that quick sort is just as bad as say selection sort, insertion sort and/or bubble sort. No! Its all about average case performance. The average case performance for these three is θ(n2) as well.What is the average case complexity for QS?
![Page 16: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/16.jpg)
Average Case Analysisassume prob. pivotpoint is p
See HW 22 p 86 for above conversion
![Page 17: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/17.jpg)
Average case continued
Multiplying by n
Subtracting these equations we have
![Page 18: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/18.jpg)
Average case QS continued
Assume we get,
Applying some simple math we have
Which give)
![Page 19: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/19.jpg)
Matrix Multiplication (Strassen)
Lets look at the product of two 2 by 2’s Clearly after you do the homework #26m1=(a11+a22)(b11+b22) m5=(a11+a12)b22
m2=(a11+a22)b11 m6=(a21+a11)(b11+b12)m3=a11(b12+b22) m7=(a12+a22)(b21+b22)m4=a22(b21+b11) will give the following!
![Page 20: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/20.jpg)
And the answer is
Original method 8 mult, four add/subStrassen’s method 7 mult, and 18 add/sub Hmmmm! So what’s the big deal?
![Page 21: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/21.jpg)
Big Matrices 2n by 2n
Where
C11 is the upper left hand corner of the matrix of size n/2 by n/2. The others are similarly defined. Now
m1=(A11+A22)(B11+B22)Is the sum and product of matrices
![Page 22: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/22.jpg)
Our function is then
void Strassen(int n, A,B, C)// these are nxn mats{ if (n<= threshold) computer C=AxB normally Partition A and B into eight submatrices strassen(n/2, A11+A22, B11+B22, M1); strassen(n/2, A21+A22, B11, M2) etc // making 7 recursive calls } NOT EIGHT!
![Page 23: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/23.jpg)
Complexity
T(n) = 7T(n/2) + cn2
Which is
T(n) = θ(nlg7) = O(n2.81)Using the general theorem.
The best know is Coppersmith and Winograd with a time complexity ofO(n2.376)
Why am I using big O here?
![Page 24: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/24.jpg)
Recalling General TheoremSee page 588
Assume for n>1 and n a power of b, T(1)=d
![Page 25: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/25.jpg)
Just a side note
Suppose we has 8 recursive calls instead of 7 in the above case. Then the recurrence relation would be
T(n) = 8T(n/2) + cn2
This has a complexity of what?
![Page 26: Foundations of Algorithms, Fourth Edition Richard Neapolitan, Kumarss Naimipour Chapter 2 Divide-and-Conquer](https://reader036.vdocuments.mx/reader036/viewer/2022062302/56816859550346895dde8bc3/html5/thumbnails/26.jpg)
When not to use divide and conquer
• An instance of size n is divided into two or more instances each almost of size n.
• An instance of size n is divided into almost
n instances of size n/c, where c is a constant