CS 312: Algorithm Analysis

Lecture #6: Divide and Conquer

This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.

Slides by: Eric Ringger, with contributions from Mike Jones, Eric Mercer, Sean Warnick

Announcements Project #1

Due: today (5pm)

Holiday: Monday

Project #2 Instructions and software distribution available from the schedule We do have one improvement to release later today, though Help session on Tuesday

5pm in 1066 TMCB

HW #4 due Wednesday

Objectives

Introduce Divide and Conquer

Apply to Multiplication

Analyze by defining a recurrence relation

Introduce the Convex Hull problem

Thought

“Nothing is particularly hard if you divide it into small jobs.”

-- Henry Ford

Divide and Conquer

... ... ...Solve the smaller pieces.

Recall Efficiency of Multiplication Algorithms

Problem: Multiplication Algorithms:

American British a la Francaise / Russe Arabic …

Efficiency so far:

Divide and Conquer

05025001

05 50 shift 4 = 250000005 01 shift 2 = 50002 50 shift 2 = 1000002 01 shift 0 = 2 2510502

Divide and Conquer

05025001

05 50 shift 4 = 250000005 01 shift 2 = 50002 50 shift 2 = 1000002 01 shift 0 = 2 2510502

Is it correct?

Divide and Conquer

05025001

05 x 50 shift 4 = 250000005 x 01 shift 2 = 50002 x 50 shift 2 = 1000002 x 01 shift 0 = 2 2510502

5001 x 502 10002 025005002510502

Is it correct?

How long does it take on an input of size n? Θ(𝑛2)

Is it Faster?

No.

Can we do better?

Yes. We shall see …

Divide and Conquer

05025001

05 x 50 shift 4 = 250000005 x 01 shift 2 = 50002 x 50 shift 2 = 1000002 x 01 shift 0 = 2 2510502

xw

y z

wy104 +wz102 +xy102 +xz100

product = wy104 + (wz + xy)102 + xz

Divide and Conquer

05025001

05 x 50 shift 4 = 250000005 x 01 shift 2 = 50002 x 50 shift 2 = 1000002 x 01 shift 0 = 2 2510502

xw

y z

wy104 +wz102 +xy102 +xz100

product = wy104 + (wz + xy)102 + xz

Divide and Conquer05025001

xw

y z

product = wy104 + (wz + xy)102 + xz

Let r = (w + x)(y + z) = wy + wz + xy + xzThen wz + xy = r – wy - xz

product = wy104 + (r – wy - xz)102 + xz

= wy104 + ((w+x)(y+z)– wy - xz)102 + xz

Multiply only three timesOriginalproduct = wy104 + (wz + xy)102 + xz100

p = wyq = xzr = (w + x)(y + z)

p104 + (r-p-q)102 + q

= wy104 + (wy + wz + xy + xz - wy - xz)102 + xz= wy104 + (wz + xy)102 + xz

Recursive application

... ... ...

Algorithm

How long does it take? Characterize running time using a recurrence

relation. Let be the amount of time to compute an

answer on an input of size . Then: *

where , the cost of dividing and reassembling sub-results

Answering a question with a question!

* Actually: -- solution in same asymptotic order of growth

Master Theorem:Analysis of Divide & Conquer

Define: = number of sub-instances that must be solved = original instance size (variable) *

= size of sub-instances= polynomial order of , where = cost of dividing and recombining

( ) ( / ) ( ) for ( )( )t n a t n b g n g n n

log

( ) log( ) ( log ) log

( ) logb

d db

d db

a db

n if a b d at n n n if a b d a

n if a b d a

Then:

We’ll prove this! * Assume that is a power of .

Analysis of DC Multiplicationwith Karatsuba’s Insight

a = 3 there are 3 instances of multiplicationb = 2 each is 1/2 the size of the original multiplication d = 1 the divide and the reassemble has linear complexity

( ) 3 ( / 2) ( )t n t n g n

Analysis of DC Multiplicationwith Karatsuba’s Insight

2log 31 1.585since a=3 2 , ( ) ( ) ( )db t n n n

a = 3 there are 3 instances of multiplicationb = 2 each is 1/2 the size of the original multiplication d = 1 the divide and the reassemble has linear complexity

( ) 3 ( / 2) ( )t n t n g n

Another Logarithm Identity

Consequence:

Essentials ofDivide and Conquer

Given problem instance of size Divide into sub-instances of size Recursively solve these sub-instances Appropriately combine their answers

Analyze with the Master Theorem

General: Divide and Conquer

function DC (x) : answer if length(x) < threshold then return adhoc(x) decompose x into a sub-instances x1, x2 … xa of size n/b for i 1 to a do yi DC(xi) recombine the yi’s to get a solution y for x return y

Where : adhoc(x) = is the basic algorithm for small instancesa = the number of divisions at each leveln/b = the fraction of the whole for a sub-instance

Choosing a Threshold Threshold = t means:

“stop dividing when problem size = t”

Threshold depends on Algorithm and implementation and platform

If t is too big? If t is too small?

Recursive vs. Iterative Algorithms Tail Recursion can be written iteratively easily

When results of recursive call aren’t used in an assignment

In fact, all recursive algorithms can be written as iteration. How? By maintaining your own stack

Is the iterative implementation better? Saves on stack allocation and function call

Is the recursive implementation better? More “elegant” to some tastes:

more natural, more readable, more maintainable.

Same algorithmic complexity

Convex Hull (Proj. #2)

Convexity

Convexity

Assignment

Read: Recurrence Relations Notes, Part 1

HW #4: 1.27, 2.1, 2.5(a-e) using the Master Theorem Due Wednesday

Extra Slides

Possible DC Solution: Step 1

Given set of n points Divide into two subsets

L containing the leftmost points R containing the rightmost points

All points with same x coordinate Assign to the same subset Even if this makes the division not exactly

into halves

Possible DC Solution: Step 2

Compute the convex hulls of L and R recursively

Possible DC Solution: Step 3 Combine the left hull CH(L) and the right hull CH(R):

Find the two edges known as the upper and lower common tangents (shown in red)

Common tangent: a line segment in the exterior of both polygons intersecting each polygon at a single vertex or a single edge.

The line containing the common tangent does not intersect the interior of either polygon

Possible DC Solution: Tips Find the upper common tangent:

Scan around the left hull in a clockwise direction and around the right hull in a counter-clockwise direction

Come up with the details of finding the common tangents – hints available in the guidelines document

The tangents divide each hull into two pieces Delete the right edges belonging to the left hull and the

left edges belonging to the right hull