young cs 331 d&a of algo. topic: divide and conquer1 divide-and-conquer general idea: divide a...
Post on 18-Jan-2018
215 Views
Preview:
DESCRIPTION
TRANSCRIPT
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 1
Divide-and-Conquer
General idea:
Divide a problem into subprograms of the same kind; solve subprograms using the same approach, and combine partial solution (if necessary).
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 2
The straightforward algorithm:
max ← min ← A (1);for i ← 2 to n do
if A (i) > max, max ← A (i);if A (i) < min, min ← A (i);
Key comparisons: 2(n – 1)
1. Find the maximum and minimum
The problem: Given a list of unordered n elements, find max and min
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 3
List List 1 List 2n elements n/2 elements n/2 elements
min, max min1, max1 min2, max2
min = MIN ( min1, min2 )max = MAX ( max1, max2)
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 4
The Divide-and-Conquer algorithm: procedure Rmaxmin (i, j, fmax, fmin); // i, j are index #, fmax,
begin // fmin are output parameters case:
i = j: fmax ← fmin ← A (i);i = j –1: if A (i) < A (j) then fmax ← A (j);
fmin ← A (i); else fmax ← A (i);
fmin ← A (j);else: mid ← ;
call Rmaxmin (i, mid, gmax, gmin);call Rmaxmin (mid+1, j, hmax, hmin);fmax ← MAX (gmax, hmax);fmin ← MIN (gmin, hmin);
endend;
2ji
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 5
Example: find max and min in the array: 22, 13, -5, -8, 15, 60, 17, 31, 47 ( n = 9 )
Index: 1 2 3 4 5 6 7 8 9Array: 22 13 -5 -8 15 60 17 31 47
Rmaxmin(1, 9, 60, -8)
1, 5, 22, -8 6, 9, 60, 17
1, 3, 22, -5 4, 5, 15, -8 6,7, 60, 17 8, 9, 47, 31
1,2, 22, 13 3, 3,-5, -5
(1)
(2)
(3) (4)
(5)
(6)
(7) (8)
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 6
Analysis: For algorithm containing recursive calls, we can use recurrence relation to find its complexity
T(n) - # of comparisons needed for RmaxminRecurrence relation:
otherwisenTnT
nnTnnT
2)2
(2)(
21)(10)(
222)
2(222)2)
8(2(2
22)2
(22)2)4
(2(2
2)2
(2)(
233
322
22
2
nTnT
nTnT
nTnT
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 7
Assume n = 2k for some integer k
25.1
212
)22()2(2
)222()2
(2
1
1211
1
n
nnT
nT
kk
kkk
k
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 8
Theorem:
Claim:
1)(
1)(
nbncnaT
nbnT where are a, b, c constants
They don’t have to be the same constant. We make them the same here for simplicity. It will not affect the overall result.
canO
cannLogOcanO
nTaLogc
c
)(
)()(
)(
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 9
Proof: Assume n = ck
bncnTanT )()(
) ) ( (
) ) ( ) ( ) (( ) (
) ) ( ) ( ) ((
) ) ( ) ( ) ((
) ( ) (
) 1 ( ) (
) ) ( (
) ( ) ) ( ( ) (
0
0 2 1
0 2 1
0 2 1
0
0
2
2
1
1
2
2
3 3
2 3 2
2 2
2
i k
i
k k k
k k k
k
k k k
k
k
k
k
k k
c
a bn
c
a
c
a
c
a bn c
a bn
c
a
c
a
c
a bn c
n b a
c
a
c
a
c
a bn b a
c
a
c
a
c
a bn c
n T a
c
a
c
a bn c
n T a
bn c
n ab c
n b c
n T a a
bn c
n ab c
n T a bn c
n b c
n T a a n T
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 10
If a < c, is a constant
T(n) = bn · a constant T(n) = O(n)
if a = c,
ik
i ca )(
0
1)(0
kca i
k
i
)()1(
)1(1)(0
nLognOnLogbn
kbnbnnT
c
k
i
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 11
If a > c, 1)(
1)()(
1
0
ca
ca
ca
k
ik
i
)(
)(
1)(
1)()()(
1
0
aLog
aLog
nLogkk
k
ik
i
c
c
c
nO
nb
ababcabn
ca
ca
bncabnnT
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 12
2. Matrix multiplication
The problem:
Multiply two matrices A and B, each of size nn
nnnnnn
CBA
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 13
The traditional way:
use three for-loop
kj
n
kikij BAC
1
)()( 3nOnT
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 14
The Divide-and-Conquer way:
2221
1211
2221
1211
2221
1211
BBB
BB
AAA
AA
CCC
CC
2222122122
2122112121
2212121112
2112111111
BABAC
BABAC
BABAC
BABAC
transform the problem of multiplying A and B, each of size [n×n] into 8 subproblems, each of size
22nn
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 15
)(
)2
(8)(
3
2
nO
annTnT
which an2 is for addition
so, it is no improvement compared with the traditional way
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 16
111111111
111111111Example:
use Divide-and-Conquer way to solve it as following:
0000033303330333
0000011101110111
0000011101110111
0003
0001
0001
0101
0011
0033
0011
0001
1111
0011
0303
0001
0101
0101
1111
3333
0011
0101
1111
1111
22
21
12
11
C
C
C
C
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 17
Strassen’s matrix multiplication: Discover a way to compute the Cij’s using 7 multiplications
and 18 additions or subtractions
))(()(
)()(
)())((
12111121
221211
112122
221211
112221
22112211
BBAAUBAATBBASBBAR
BAAQBBAAP
UQRPCSQCTRC
VTSPCBBAAV
22
21
12
11
22212212 ))((
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 18
•Algorithm :
procedure Strassen (n, A, B, C) // n is size, A,B the input matrices, C output matrix
beginif n = 2,
else (cont.)
;;;;
2222122122
2122112121
2212121112
2112111111
babaCbabaCbabaCbabaC
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 19
elsePartition A into 4 submatrices: ;Partition B into 4 submatrices: ; call Strassen ( ; call Strassen ( call Strassen ( ;
call Strassen ( ;
call Strassen ( ;
22211211 ,,, AAAA22211211 ,,, BBBB
),,,2 22112211 PBBAAn
);,,,2 112221 QBAAn
),,,2 221211 RBBAn
),,,2 112122 SBBAn
),,,2 221211 TBAAn
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 20
call Strassen ( ;
call Strassen ( ;
end;
),,,2 12111121 UBBAAn
;;;
;
22
21
12
11
UQRPCSQCTRC
VTSPC
),,,2 22212212 VBBAAn
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 21
Analysis:
2
2)2
(7)(2
nb
nannTnT
2222
33
222
2
2
)47()
47()
2(7
)47()
2(7
)2
(7)(
ananannT
anannT
annTnT
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 22
Assume n = 2k for some integer k
)()(
)(
)(7)47(7
)47(7
147
1)47(
7
1)47()
2(7
81.27
777
47
22
2
1
21
221
1
nOnO
ncbcnnb
ncnbcnb
ncb
anb
annT
Lg
LgLgLg
LgLgnLgnLgn
kk
k
k
kk
k
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 23
3. Integer multiplication
The problem:
Multiply two large integers (n digits)
The traditional way:
Use two loops, it takes O(n2) operations
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 24
The Divide-and-Conquer way:
Suppose x and y are large integers, divide x into two parts (a and b), and divide y into c and d.
x:
y:
a b baxn
210
c d dcyn
210
)(1010
)10)(10(
2
22
bcadbdac
dcbayxn
n
nn
So, transform the problem of multiplying two n-digit integers into four subproblems of multiplying two -digit integers.
2n
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 25
Worst-Case is:
However, it is same as the traditional way.
Instead, we write:
Worst-Case is:
)()(
)2
(4)(
242 nOnO
bnnTnT
Log
)()(
)2
(3)(
58.132 nOnO
bnnTnT
Log
)))(((1010
)(1010
2
2
bdacdcbabdac
bcadbdacyxn
n
nn
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 26
The algorithm by Divide-and-Conquer: function multiplication (x,y)
beginn = MAX ( # of digits in x, # of digits in y);if (x = 0) or (y = 0), return 0;else if (n = 1), return x*y in the usual way;
else m = ; a = x divide 10m; b = x rem 10m; c = y divide 10m; d = y rem 10m; p1= MULTIPLICATION (a, c); p2= MULTIPLICATION (b, d);
p3 = MULTIPLICATION (a+b, c+d); return ;
end;
2n
)(1010 21322
1 ppppp mm
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 27
4. Merge Sort The problem:
Given a list of n numbers, sort them in non-decreasing order
idea: Split each part into 2 equal parts and sort each part using Mergesort, then merge the two sorted sub-lists into one sorted list.
The algorithm:
procedure MergeSort (low, high)begin
if then
call MergeSort (low, mid); call MergeSort (mid+1, high); call Merge (low, mid, high);
end;
highlow
;2
highlowmid
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 28
procedure Merge (low, mid, high)begin
while (i mid and j high) if A(i) < A(j), then U(k) A(i);
i++;else U(k) A(j);
j++;k++;
if (i > mid) move A(j) through A(high) to U(k) through U(high);else
move A(i) through A(mid) to U(k) through U(high);for p low to high A(p) U(p);
end;
;,1, lowkmidjlowi
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 29
Analysis:Worst-Case for MergeSort is:
Best-Case & Average-Case? Merge sort pays no attention to the original order of the list, it will keep divide the list into half until sub-lists of length 1, then start merging.
Therefore Best-Case & Average-Case are the same as the Worst-Case. O(nLogn)
)(
)2
(2)(
nLognO
bnnTnT
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 30
5. Quick Sort
The problem:
Same as Merge Sort
Compared with Merge Sort:
• Quick Sort also use Divide-and-Conquer strategy
• Quick Sort eliminates merging operations
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 31
How it works?
Use partition
Eg: Sort array 65, 70, 75, 80, 85, 60, 55, 50, 4565 70 75 80 85 60 55 50 45
60 55 50 45 70 75 80 85
55 50 45
50 45
45
75 80 85
80 85
85
Pivot item
65
60 65 70
55 60 65 70 75
55 60 65 70 75 80
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 32
The algorithm:procedure QuickSort (p, q)
beginif p < q,then call Partition (p, q, pivotposition);
call QuickSort (p, pivotposition –1);call QuickSort (pivotposition+1, q);
end;
procedure Partition (low, high, pivotposition)begin
v A(low);j low;for i (low + 1) to high
if A(i) < v,j++;A(i) A(j);
pivotposition j;A(low) A(pivotposition);
end;
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 33
Analysis:
Worst-Case:Call Partition O(n) times, each time takes O(n) steps. So O(n2), and it is worse than Merge Sort in the Worst-Case.
Best-Case:
Split the list evenly each time.
)(nLognO
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 34
Average-Case: Assume pivot is selected randomly, so the probability of
pivot being the kth element is equal, k.
C(n) = # of key comparison for n items
• average it over all k, (CA(n) is average performance)
• multiply both side by n
kn
kpivotprob ,1)(
)1())1()1()0((2)1()(
nCCCnnnCn
AAA
A
)()1()1( knCkCn
))()1((1)1()(1
knCkCn
nnC A
n
kAA
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 35
• replace n by n-1
• subtract (2) from (1)
)2())2()0((2)2)(1()1()1(
nCCnnnCn
AA
A
)1()1()1(2)()1(2)1(2)1()1()(
nCnnnCnnCnnCnnCn
AA
AAA
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 36
• divide n(n + 1) for both sides
Best-Case = Average-Case
)()()(
)1(21212
)1(120
))1(
143
232
1(22
)1(0)1(:
)1()1(2
)1()2(2
)1()2()3(2
2)3(
)1()1(2
)1()2(2
1)2(
)1()1(2)1(
1)(
12
2
nLognOnCLognO
LognLogdkkk
kkk
nnnC
CNote
nnn
nnn
nnn
nnC
nnn
nnn
nnC
nnn
nnC
nnC
A
ee
nn
k
n
k
A
A
A
A
AA
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 37
6. Selection The problem:
Given a list of n elements find the kth smallest oneNote: special cases: when k = 1, min
when k = n, max
The solving strategy: If use sorting strategy like MergeSort,
Both Best-Case and Worst-Case take O(n Log n)
If use Quick Sort,
Best-Case is O(n) (when pivot is the kth smallest one)Worst-Case will call partition O(n) times, each time takes O(n), so Worst-Case is O(n2).
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 38
Select1 – the first algorithm
idea: use “partition” (from Quick Sort) repeatedly until we find the kth element.
For each iteration:
If pivot position = k Done!
If pivot position < k to select (k-i)th element in the 2nd sub-list.
If pivot position > k to select kth element in the 1st sub-list.
pivot
i = pivot position
1st sub-list 2nd sub-list
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 39
procedure Select1 (A, n, k) // A is array, n is # of array, k is keybegin
m 1;j n;loop
call Partition (m, j, pivotposition);case:
k = pivotposition:return A(k);
k < pivotposition:j pivotposition – 1;
else:m pivotposition + 1;
end loop;end;
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 40
By choosing pivot more carefully, we can obtain a selection algorithm with Worst-Case complexity O(n)
How: Make sure pivot v is chosen s.t. at least some fraction of the elements will be smaller than v and at least some other fraction of elements will be greater than v.
mm (median of medians)Non-decreasingOrder(all columns) The middle row is in
non-decreasing order
Elements mm
Elements mm
Example: 35 (n) elements divide into 7 (n/r) subsets of size 5 (r)
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 41
By choosing pivot more carefully, we can obtain a selection algorithm with Worst-Case complexity O(n)
Use mm (median of medians) rule to choose pivot
procedure Select2 (A, n, k) begin
if n r, then sort A and return the kth element;
divide A into subset of size r each, ignore excess
elements, and let be the set of
medians of the subsets;
rn
rnmmmM ,,, 21
rn
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 42
Select2 ;
use “Partition” to partition A using v as the pivot;// Assume v is at pivotposition
case: k = pivotposition: return (v);
k < pivotposition: let S be the set of elements A (1,…, pivotposition –1),
return Select2 (S, pivotposition –1, k);
else: let R be the set of element A (pivotposition+1,…, n),
return Select2 (R, n- pivotposition, k-pivotposition); end case; end;
v )2,,(
rn
rnM
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 43
Analysis:• How many Mi’s (or ) mm?
At least
• How many elements (or ) mm?
At least
• How many elements > mm (or < mm)? At most
Assume r = 5
2
rn
22 r
nr
)22
(
rnrn
)24(75.0
2.17.05
45.1
55.1)2
53(
nn
nnn
nnnn
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 44
• Therefore, procedure Select2 the Worst-Case complexity is:
where c is chosen sufficiently large so that T(n) cn for n < 24.
Proof:
Use induction to show T(n) 20 cn (n 24)
IB (induction base):
cnnTnTnT )43()
5()(
cnccncn
cTTnTn
2024
24)2443()
524()(,24
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 45
IH (induction hypothesis):
Suppose
IS (induction solution):
When n = m;
T(n) = O(n) complexity of Select2 of n elements
mncnnT 2420)(
cn
cmmcmc
cmmTmTmT
204320
520
)43()
5()(
top related