young cs 331 d&a of algo. topic: divide and conquer1 divide-and-conquer general idea: divide a...

YoungCS 331 D&A of Algo.

Topic: Divide and Conquer 1

Divide-and-Conquer

General idea:

Divide a problem into subprograms of the same kind; solve subprograms using the same approach, and combine partial solution (if necessary).



The straightforward algorithm:

max ← min ← A (1);for i ← 2 to n do

if A (i) > max, max ← A (i);if A (i) < min, min ← A (i);

Key comparisons: 2(n – 1)

1. Find the maximum and minimum

The problem: Given a list of unordered n elements, find max and min



List List 1 List 2n elements n/2 elements n/2 elements

min, max min1, max1 min2, max2

min = MIN ( min1, min2 )max = MAX ( max1, max2)



The Divide-and-Conquer algorithm: procedure Rmaxmin (i, j, fmax, fmin); // i, j are index #, fmax,

begin // fmin are output parameters case:

i = j: fmax ← fmin ← A (i);i = j –1: if A (i) < A (j) then fmax ← A (j);

fmin ← A (i); else fmax ← A (i);

fmin ← A (j);else: mid ← ;

call Rmaxmin (i, mid, gmax, gmin);call Rmaxmin (mid+1, j, hmax, hmin);fmax ← MAX (gmax, hmax);fmin ← MIN (gmin, hmin);

endend;

2ji



Example: find max and min in the array: 22, 13, -5, -8, 15, 60, 17, 31, 47 ( n = 9 )

Index: 1 2 3 4 5 6 7 8 9Array: 22 13 -5 -8 15 60 17 31 47

Rmaxmin(1, 9, 60, -8)

1, 5, 22, -8 6, 9, 60, 17

1, 3, 22, -5 4, 5, 15, -8 6,7, 60, 17 8, 9, 47, 31

1,2, 22, 13 3, 3,-5, -5

(1)

(2)

(3) (4)

(5)

(6)

(7) (8)



Analysis: For algorithm containing recursive calls, we can use recurrence relation to find its complexity

T(n) - # of comparisons needed for RmaxminRecurrence relation:

otherwisenTnT

nnTnnT

2)2

(2)(

21)(10)(

222)

2(222)2)

8(2(2

22)2

(22)2)4

(2(2

2)2

(2)(

233

322

22

2

nTnT

nTnT

nTnT



Assume n = 2k for some integer k

25.1

212

)22()2(2

)222()2

(2

1

1211

1

n

nnT

nT

kk

kkk

k



Theorem:

Claim:

1)(

1)(

nbncnaT

nbnT where are a, b, c constants

They don’t have to be the same constant. We make them the same here for simplicity. It will not affect the overall result.

canO

cannLogOcanO

nTaLogc

c

)(

)()(

)(



Proof: Assume n = ck

bncnTanT )()(

) ) ( (

) ) ( ) ( ) (( ) (

) ) ( ) ( ) ((

) ) ( ) ( ) ((

) ( ) (

) 1 ( ) (

) ) ( (

) ( ) ) ( ( ) (

0

0 2 1

0 2 1

0 2 1

0

0

2

2

1

1

2

2

3 3

2 3 2

2 2

2

i k

i

k k k

k k k

k

k k k

k

k

k

k

k k

c

a bn

c

a

c

a

c

a bn c

a bn

c

a

c

a

c

a bn c

n b a

c

a

c

a

c

a bn b a

c

a

c

a

c

a bn c

n T a

c

a

c

a bn c

n T a

bn c

n ab c

n b c

n T a a

bn c

n ab c

n T a bn c

n b c

n T a a n T



If a < c, is a constant

T(n) = bn · a constant T(n) = O(n)

if a = c,

ik

i ca )(

0

1)(0

kca i

k

i

)()1(

)1(1)(0

nLognOnLogbn

kbnbnnT

c

k

i



If a > c, 1)(

1)()(

1

0

ca

ca

ca

k

ik

i

)(

)(

1)(

1)()()(

1

0

aLog

aLog

nLogkk

k

ik

i

c

c

c

nO

nb

ababcabn

ca

ca

bncabnnT



2. Matrix multiplication

The problem:

Multiply two matrices A and B, each of size nn

nnnnnn

CBA



The traditional way:

use three for-loop

kj

n

kikij BAC

1

)()( 3nOnT



The Divide-and-Conquer way:

2221

1211

2221

1211

2221

1211

BBB

BB

AAA

AA

CCC

CC

2222122122

2122112121

2212121112

2112111111

BABAC

BABAC

BABAC

BABAC

transform the problem of multiplying A and B, each of size [n×n] into 8 subproblems, each of size

22nn



)(

)2

(8)(

3

2

nO

annTnT

which an2 is for addition

so, it is no improvement compared with the traditional way



111111111

111111111Example:

use Divide-and-Conquer way to solve it as following:

0000033303330333

0000011101110111

0000011101110111

0003

0001

0001

0101

0011

0033

0011

0001

1111

0011

0303

0001

0101

0101

1111

3333

0011

0101

1111

1111

22

21

12

11

C

C

C

C



Strassen’s matrix multiplication: Discover a way to compute the Cij’s using 7 multiplications

and 18 additions or subtractions

))(()(

)()(

)())((

12111121

221211

112122

221211

112221

22112211

BBAAUBAATBBASBBAR

BAAQBBAAP

UQRPCSQCTRC

VTSPCBBAAV

22

21

12

11

22212212 ))((



•Algorithm :

procedure Strassen (n, A, B, C) // n is size, A,B the input matrices, C output matrix

beginif n = 2,

else (cont.)

;;;;

2222122122

2122112121

2212121112

2112111111

babaCbabaCbabaCbabaC



elsePartition A into 4 submatrices: ;Partition B into 4 submatrices: ; call Strassen ( ; call Strassen ( call Strassen ( ;

call Strassen ( ;

call Strassen ( ;

22211211 ,,, AAAA22211211 ,,, BBBB

),,,2 22112211 PBBAAn

);,,,2 112221 QBAAn

),,,2 221211 RBBAn

),,,2 112122 SBBAn

),,,2 221211 TBAAn



call Strassen ( ;

call Strassen ( ;

end;

),,,2 12111121 UBBAAn

;;;

;

22

21

12

11

UQRPCSQCTRC

VTSPC

),,,2 22212212 VBBAAn



Analysis:

2

2)2

(7)(2

nb

nannTnT

2222

33

222

2

2

)47()

47()

2(7

)47()

2(7

)2

(7)(

ananannT

anannT

annTnT



Assume n = 2k for some integer k

)()(

)(

)(7)47(7

)47(7

147

1)47(

7

1)47()

2(7

81.27

777

47

22

2

1

21

221

1

nOnO

ncbcnnb

ncnbcnb

ncb

anb

annT

Lg

LgLgLg

LgLgnLgnLgn

kk

k

k

kk

k



3. Integer multiplication

The problem:

Multiply two large integers (n digits)

The traditional way:

Use two loops, it takes O(n2) operations



The Divide-and-Conquer way:

Suppose x and y are large integers, divide x into two parts (a and b), and divide y into c and d.

x:

y:

a b baxn

210

c d dcyn

210

)(1010

)10)(10(

2

22

bcadbdac

dcbayxn

n

nn

So, transform the problem of multiplying two n-digit integers into four subproblems of multiplying two -digit integers.

2n



Worst-Case is:

However, it is same as the traditional way.

Instead, we write:

Worst-Case is:

)()(

)2

(4)(

242 nOnO

bnnTnT

Log

)()(

)2

(3)(

58.132 nOnO

bnnTnT

Log

)))(((1010

)(1010

2

2

bdacdcbabdac

bcadbdacyxn

n

nn



The algorithm by Divide-and-Conquer: function multiplication (x,y)

beginn = MAX ( # of digits in x, # of digits in y);if (x = 0) or (y = 0), return 0;else if (n = 1), return x*y in the usual way;

else m = ; a = x divide 10m; b = x rem 10m; c = y divide 10m; d = y rem 10m; p1= MULTIPLICATION (a, c); p2= MULTIPLICATION (b, d);

p3 = MULTIPLICATION (a+b, c+d); return ;

end;

2n

)(1010 21322

1 ppppp mm



4. Merge Sort The problem:

Given a list of n numbers, sort them in non-decreasing order

idea: Split each part into 2 equal parts and sort each part using Mergesort, then merge the two sorted sub-lists into one sorted list.

The algorithm:

procedure MergeSort (low, high)begin

if then

call MergeSort (low, mid); call MergeSort (mid+1, high); call Merge (low, mid, high);

end;

highlow

;2

highlowmid



procedure Merge (low, mid, high)begin

while (i mid and j high) if A(i) < A(j), then U(k) A(i);

i++;else U(k) A(j);

j++;k++;

if (i > mid) move A(j) through A(high) to U(k) through U(high);else

move A(i) through A(mid) to U(k) through U(high);for p low to high A(p) U(p);

end;

;,1, lowkmidjlowi



Analysis:Worst-Case for MergeSort is:

Best-Case & Average-Case? Merge sort pays no attention to the original order of the list, it will keep divide the list into half until sub-lists of length 1, then start merging.

Therefore Best-Case & Average-Case are the same as the Worst-Case. O(nLogn)

)(

)2

(2)(

nLognO

bnnTnT



5. Quick Sort

The problem:

Same as Merge Sort

Compared with Merge Sort:

• Quick Sort also use Divide-and-Conquer strategy

• Quick Sort eliminates merging operations



How it works?

Use partition

Eg: Sort array 65, 70, 75, 80, 85, 60, 55, 50, 4565 70 75 80 85 60 55 50 45

60 55 50 45 70 75 80 85

55 50 45

50 45

45

75 80 85

80 85

85

Pivot item

65

60 65 70

55 60 65 70 75

55 60 65 70 75 80



The algorithm:procedure QuickSort (p, q)

beginif p < q,then call Partition (p, q, pivotposition);

call QuickSort (p, pivotposition –1);call QuickSort (pivotposition+1, q);

end;

procedure Partition (low, high, pivotposition)begin

v A(low);j low;for i (low + 1) to high

if A(i) < v,j++;A(i) A(j);

pivotposition j;A(low) A(pivotposition);

end;



Analysis:

Worst-Case:Call Partition O(n) times, each time takes O(n) steps. So O(n2), and it is worse than Merge Sort in the Worst-Case.

Best-Case:

Split the list evenly each time.

)(nLognO



Average-Case: Assume pivot is selected randomly, so the probability of

pivot being the kth element is equal, k.

C(n) = # of key comparison for n items

• average it over all k, (CA(n) is average performance)

• multiply both side by n

kn

kpivotprob ,1)(

)1())1()1()0((2)1()(

nCCCnnnCn

AAA

A

)()1()1( knCkCn

))()1((1)1()(1

knCkCn

nnC A

n

kAA



• replace n by n-1

• subtract (2) from (1)

)2())2()0((2)2)(1()1()1(

nCCnnnCn

AA

A

)1()1()1(2)()1(2)1(2)1()1()(

nCnnnCnnCnnCnnCn

AA

AAA



• divide n(n + 1) for both sides

Best-Case = Average-Case

)()()(

)1(21212

)1(120

))1(

143

232

1(22

)1(0)1(:

)1()1(2

)1()2(2

)1()2()3(2

2)3(

)1()1(2

)1()2(2

1)2(

)1()1(2)1(

1)(

12

2

nLognOnCLognO

LognLogdkkk

kkk

nnnC

CNote

nnn

nnn

nnn

nnC

nnn

nnn

nnC

nnn

nnC

nnC

A

ee

nn

k

n

k

A

A

A

A

AA



6. Selection The problem:

Given a list of n elements find the kth smallest oneNote: special cases: when k = 1, min

when k = n, max

The solving strategy: If use sorting strategy like MergeSort,

Both Best-Case and Worst-Case take O(n Log n)

If use Quick Sort,

Best-Case is O(n) (when pivot is the kth smallest one)Worst-Case will call partition O(n) times, each time takes O(n), so Worst-Case is O(n2).



Select1 – the first algorithm

idea: use “partition” (from Quick Sort) repeatedly until we find the kth element.

For each iteration:

If pivot position = k Done!

If pivot position < k to select (k-i)th element in the 2nd sub-list.

If pivot position > k to select kth element in the 1st sub-list.

pivot

i = pivot position

1st sub-list 2nd sub-list



procedure Select1 (A, n, k) // A is array, n is # of array, k is keybegin

m 1;j n;loop

call Partition (m, j, pivotposition);case:

k = pivotposition:return A(k);

k < pivotposition:j pivotposition – 1;

else:m pivotposition + 1;

end loop;end;



By choosing pivot more carefully, we can obtain a selection algorithm with Worst-Case complexity O(n)

How: Make sure pivot v is chosen s.t. at least some fraction of the elements will be smaller than v and at least some other fraction of elements will be greater than v.

mm (median of medians)Non-decreasingOrder(all columns) The middle row is in

non-decreasing order

Elements mm

Elements mm

Example: 35 (n) elements divide into 7 (n/r) subsets of size 5 (r)



By choosing pivot more carefully, we can obtain a selection algorithm with Worst-Case complexity O(n)

Use mm (median of medians) rule to choose pivot

procedure Select2 (A, n, k) begin

if n r, then sort A and return the kth element;

divide A into subset of size r each, ignore excess

elements, and let be the set of

medians of the subsets;

rn

rnmmmM ,,, 21

rn



Select2 ;

use “Partition” to partition A using v as the pivot;// Assume v is at pivotposition

case: k = pivotposition: return (v);

k < pivotposition: let S be the set of elements A (1,…, pivotposition –1),

return Select2 (S, pivotposition –1, k);

else: let R be the set of element A (pivotposition+1,…, n),

return Select2 (R, n- pivotposition, k-pivotposition); end case; end;

v )2,,(

rn

rnM



Analysis:• How many Mi’s (or ) mm?

At least

• How many elements (or ) mm?

At least

• How many elements > mm (or < mm)? At most

Assume r = 5

2

rn

22 r

nr

)22

(

rnrn

)24(75.0

2.17.05

45.1

55.1)2

53(

nn

nnn

nnnn



• Therefore, procedure Select2 the Worst-Case complexity is:

where c is chosen sufficiently large so that T(n) cn for n < 24.

Proof:

Use induction to show T(n) 20 cn (n 24)

IB (induction base):

cnnTnTnT )43()

5()(

cnccncn

cTTnTn

2024

24)2443()

524()(,24



IH (induction hypothesis):

Suppose

IS (induction solution):

When n = m;

T(n) = O(n) complexity of Select2 of n elements

mncnnT 2420)(

cn

cmmcmc

cmmTmTmT

204320

520

)43()

5()(

young cs 331 d&a of algo. topic: divide and conquer1 divide-and-conquer general idea: divide a...

Documents