Divide and Conquer

June 4, 2014

Divide and Conquer

Divide the problem into a number of subproblems

Conquer the subproblems by solving them recursively or ifthey are small, there must be an easy solution.

Combine the solutions to the subproblems to the solution ofthe problem

Divide and Conquer

Divide the problem into a number of subproblems

Conquer the subproblems by solving them recursively or ifthey are small, there must be an easy solution.

Combine the solutions to the subproblems to the solution ofthe problem

Divide and Conquer

Divide the problem into a number of subproblems

Conquer the subproblems by solving them recursively or ifthey are small, there must be an easy solution.

Combine the solutions to the subproblems to the solution ofthe problem

Divide and Conquer

1 Integer Multiplication.

2 Finding a closest pair of points in 2D plane.

3 Matrix Multiplication

Divide and Conquer

We are given two n-bits numbers x , y and we want to computex × y .

The usual multiplication takes O(n2) times.

Can we do better than O(n2) ?

We can write x = x12n2 + x0 and y = y12

n2 + y0.

x0, x1, y0, y1 are n2 -bits numbers.

Divide and Conquer

We are given two n-bits numbers x , y and we want to computex × y .

The usual multiplication takes O(n2) times.

Can we do better than O(n2) ?

We can write x = x12n2 + x0 and y = y12

n2 + y0.

x0, x1, y0, y1 are n2 -bits numbers.

Divide and Conquer

We are given two n-bits numbers x , y and we want to computex × y .

The usual multiplication takes O(n2) times.

Can we do better than O(n2) ?

We can write x = x12n2 + x0 and y = y12

n2 + y0.

x0, x1, y0, y1 are n2 -bits numbers.

Divide and Conquer

We are given two n-bits numbers x , y and we want to computex × y .

The usual multiplication takes O(n2) times.

Can we do better than O(n2) ?

We can write x = x12n2 + x0 and y = y12

n2 + y0.

x0, x1, y0, y1 are n2 -bits numbers.

Divide and Conquer

For simplicity we write xy instead of x × y .

xy = (x12n2 + x0)(y12

n2 + y0) = x1y12n + (x1y0 + x0y1)2

n2 + x0y0.

Now if we compute each of the x1y1, x0y0, x1y0, x0y1 then we cancompute xy .

T (n) ≤ 4T (n/2) + cn.

O(n) is the time takes to add two n-bits numbers.

Using the Master Method for a = 4, b = 2 and f (n) = cn weobtain :

T (n) ≤ O(nlog2 4) = O(n2).

Not good!!

Divide and Conquer

For simplicity we write xy instead of x × y .

xy = (x12n2 + x0)(y12

n2 + y0) = x1y12n + (x1y0 + x0y1)2

n2 + x0y0.

Now if we compute each of the x1y1, x0y0, x1y0, x0y1 then we cancompute xy .

T (n) ≤ 4T (n/2) + cn.

O(n) is the time takes to add two n-bits numbers.

Using the Master Method for a = 4, b = 2 and f (n) = cn weobtain :

T (n) ≤ O(nlog2 4) = O(n2).

Not good!!

Divide and Conquer

For simplicity we write xy instead of x × y .

xy = (x12n2 + x0)(y12

n2 + y0) = x1y12n + (x1y0 + x0y1)2

n2 + x0y0.

Now if we compute each of the x1y1, x0y0, x1y0, x0y1 then we cancompute xy .

T (n) ≤ 4T (n/2) + cn.

O(n) is the time takes to add two n-bits numbers.

Using the Master Method for a = 4, b = 2 and f (n) = cn weobtain :

T (n) ≤ O(nlog2 4) = O(n2).

Not good!!

Divide and Conquer

For simplicity we write xy instead of x × y .

xy = (x12n2 + x0)(y12

n2 + y0) = x1y12n + (x1y0 + x0y1)2

n2 + x0y0.

Now if we compute each of the x1y1, x0y0, x1y0, x0y1 then we cancompute xy .

T (n) ≤ 4T (n/2) + cn.

O(n) is the time takes to add two n-bits numbers.

Using the Master Method for a = 4, b = 2 and f (n) = cn weobtain :

T (n) ≤ O(nlog2 4) = O(n2).

Not good!!

Divide and Conquer

Theorem

Let a ≥ 1 and b > 1 be constants, let f (n) be a function, and letT (n) be defined on the nonnegative integers by the recurrence

T (n) = aT (n/b) + f (n),

where we interpret n/b to mean either bn/bc or dn/be. Then T (n)can be bounded asymptotically as follows.

1 If f (n) = O(n(logb a)−ε) for some constant ε > 0, thenT (n) = Θ(nlogb a).

2 If f (n) = Θ(nlogb a), thenT (n) = Θ(nlogb a · log n).

3 If f (n) = Ω(n(logb a)+ε) for some constant ε > 0, and ifa · f (n/b) ≤ c · f (n) for some constant c < 1 and allsufficiently large n, then T (n) = Θ(f (n)).

Divide and Conquer

If we compute p = (x1 + x0)(y1 + y0) = x1y1 + x1y0 + x0y1 + x0y0then instead of x1y0 + x0y1 we can write p − x1y1 − x0y0.

p is the multiplication of two n2 -bits numbers.

So we need to compute the multiplication of three pairs of n2 -bits

numbers.

Divide and Conquer

If we compute p = (x1 + x0)(y1 + y0) = x1y1 + x1y0 + x0y1 + x0y0then instead of x1y0 + x0y1 we can write p − x1y1 − x0y0.

p is the multiplication of two n2 -bits numbers.

So we need to compute the multiplication of three pairs of n2 -bits

numbers.

Divide and Conquer

Recursive-Multiply(x,y)

1. x := x12n2 + x0 and y := y12

n2 + y0.

2. Compute x1 + x0 and y1 + y0

3. p := Recursive-Multiply (x0 + x1,y0 + y1)

4. x1y1 := Recursive-Multiply (x1,y1)

5. x0y0 := Recursive-Multiply (x0,y0)

6. Return x1y12n + (p − x1y1 − x0y0)2n2 + x0y0

T (n) ≤ 3T (n/2) + cn.T (n) ≤ O(nlog2 3) = O(n1.59).

Divide and Conquer

Recursive-Multiply(x,y)

1. x := x12n2 + x0 and y := y12

n2 + y0.

2. Compute x1 + x0 and y1 + y0

3. p := Recursive-Multiply (x0 + x1,y0 + y1)

4. x1y1 := Recursive-Multiply (x1,y1)

5. x0y0 := Recursive-Multiply (x0,y0)

6. Return x1y12n + (p − x1y1 − x0y0)2n2 + x0y0

T (n) ≤ 3T (n/2) + cn.T (n) ≤ O(nlog2 3) = O(n1.59).

Divide and Conquer

Smallest distance between pairs of points

We are give a set p1, p2, . . . , pn of n points in plane (2D plane).Find a pair of points with the smallest distance.

It is clear that we can solve the problem in O(n2).

Can we do better than O(n2) ?

Sort the points by x-coordinate and produce list Px (O(n log n)).

Sort the points by y -coordinate and produce list Py (O(n log n)).

Set Q be the set of points in the first n2 positions in Px and let R

be the rest of the points in Px .

Construct the lists Qx and Qy and Rx and Ry .

1. Recursively compute closest pair of points for Q, say q0, q1 and

2. Recursively compute closest pair of points for R, say r0, r1.

Divide and Conquer

Smallest distance between pairs of points

We are give a set p1, p2, . . . , pn of n points in plane (2D plane).Find a pair of points with the smallest distance.

It is clear that we can solve the problem in O(n2).

Can we do better than O(n2) ?

Sort the points by x-coordinate and produce list Px (O(n log n)).

Sort the points by y -coordinate and produce list Py (O(n log n)).

Set Q be the set of points in the first n2 positions in Px and let R

be the rest of the points in Px .

Construct the lists Qx and Qy and Rx and Ry .

1. Recursively compute closest pair of points for Q, say q0, q1 and

2. Recursively compute closest pair of points for R, say r0, r1.

Divide and Conquer

Smallest distance between pairs of points

We are give a set p1, p2, . . . , pn of n points in plane (2D plane).Find a pair of points with the smallest distance.

It is clear that we can solve the problem in O(n2).

Can we do better than O(n2) ?

Sort the points by x-coordinate and produce list Px (O(n log n)).

Sort the points by y -coordinate and produce list Py (O(n log n)).

Set Q be the set of points in the first n2 positions in Px and let R

be the rest of the points in Px .

Construct the lists Qx and Qy and Rx and Ry .

1. Recursively compute closest pair of points for Q, say q0, q1 and

2. Recursively compute closest pair of points for R, say r0, r1.

Divide and Conquer

Smallest distance between pairs of points

We are give a set p1, p2, . . . , pn of n points in plane (2D plane).Find a pair of points with the smallest distance.

It is clear that we can solve the problem in O(n2).

Can we do better than O(n2) ?

Sort the points by x-coordinate and produce list Px (O(n log n)).

Sort the points by y -coordinate and produce list Py (O(n log n)).

Set Q be the set of points in the first n2 positions in Px and let R

be the rest of the points in Px .

Construct the lists Qx and Qy and Rx and Ry .

1. Recursively compute closest pair of points for Q, say q0, q1 and

2. Recursively compute closest pair of points for R, say r0, r1.

Divide and Conquer

Smallest distance between pairs of points

We are give a set p1, p2, . . . , pn of n points in plane (2D plane).Find a pair of points with the smallest distance.

It is clear that we can solve the problem in O(n2).

Can we do better than O(n2) ?

Sort the points by x-coordinate and produce list Px (O(n log n)).

Sort the points by y -coordinate and produce list Py (O(n log n)).

Set Q be the set of points in the first n2 positions in Px and let R

be the rest of the points in Px .

Construct the lists Qx and Qy and Rx and Ry .

1. Recursively compute closest pair of points for Q, say q0, q1 and

2. Recursively compute closest pair of points for R, say r0, r1.

Divide and Conquer

Smallest distance between pairs of points

We are give a set p1, p2, . . . , pn of n points in plane (2D plane).Find a pair of points with the smallest distance.

It is clear that we can solve the problem in O(n2).

Can we do better than O(n2) ?

Sort the points by x-coordinate and produce list Px (O(n log n)).

Sort the points by y -coordinate and produce list Py (O(n log n)).

Set Q be the set of points in the first n2 positions in Px and let R

be the rest of the points in Px .

Construct the lists Qx and Qy and Rx and Ry .

1. Recursively compute closest pair of points for Q, say q0, q1 and

2. Recursively compute closest pair of points for R, say r0, r1.

Divide and Conquer

Smallest distance between pairs of points

We are give a set p1, p2, . . . , pn of n points in plane (2D plane).Find a pair of points with the smallest distance.

It is clear that we can solve the problem in O(n2).

Can we do better than O(n2) ?

Sort the points by x-coordinate and produce list Px (O(n log n)).

Sort the points by y -coordinate and produce list Py (O(n log n)).

Set Q be the set of points in the first n2 positions in Px and let R

be the rest of the points in Px .

Construct the lists Qx and Qy and Rx and Ry .

1. Recursively compute closest pair of points for Q, say q0, q1 and

2. Recursively compute closest pair of points for R, say r0, r1.

Divide and Conquer

Let δ be the smallest of d(q0, q1), d(r0, r1) (d(x , y) denote thedistance between x , y ).

Let x∗ denote the coordinates of the rightmost point in Q and letL be the line x∗ = x .L separates Q from R.

If there exist q ∈ Q and r ∈ R with d(q, r) < δ then each of q, rlies in distance at most δ of L.

So we can narrow down our search...Let S be the set of points in P in distance δ from L.

Let Sy denote the list of points in S sorted by increasingy -coordinate. (by going through list Py , we can compute Sy inO(n).)

Divide and Conquer

Let δ be the smallest of d(q0, q1), d(r0, r1) (d(x , y) denote thedistance between x , y ).

Let x∗ denote the coordinates of the rightmost point in Q and letL be the line x∗ = x .L separates Q from R.

If there exist q ∈ Q and r ∈ R with d(q, r) < δ then each of q, rlies in distance at most δ of L.

So we can narrow down our search...Let S be the set of points in P in distance δ from L.

Let Sy denote the list of points in S sorted by increasingy -coordinate. (by going through list Py , we can compute Sy inO(n).)

Divide and Conquer

Let δ be the smallest of d(q0, q1), d(r0, r1) (d(x , y) denote thedistance between x , y ).

Let x∗ denote the coordinates of the rightmost point in Q and letL be the line x∗ = x .L separates Q from R.

If there exist q ∈ Q and r ∈ R with d(q, r) < δ then each of q, rlies in distance at most δ of L.

So we can narrow down our search...Let S be the set of points in P in distance δ from L.

Let Sy denote the list of points in S sorted by increasingy -coordinate. (by going through list Py , we can compute Sy inO(n).)

Divide and Conquer

Let δ be the smallest of d(q0, q1), d(r0, r1) (d(x , y) denote thedistance between x , y ).

Let x∗ denote the coordinates of the rightmost point in Q and letL be the line x∗ = x .L separates Q from R.

If there exist q ∈ Q and r ∈ R with d(q, r) < δ then each of q, rlies in distance at most δ of L.

So we can narrow down our search...Let S be the set of points in P in distance δ from L.

Let Sy denote the list of points in S sorted by increasingy -coordinate. (by going through list Py , we can compute Sy inO(n).)

Divide and Conquer

If there exist q ∈ Q and r ∈ R for which d(q, r) < δ iff there exists, s ′ ∈ S for which d(s, s ′) < δ.

If s, s ′ ∈ S have the property that d(s, s ′) < δ then s, s ′ are within15 positions of each other in the sorted list Sy .

Divide and Conquer

If there exist q ∈ Q and r ∈ R for which d(q, r) < δ iff there exists, s ′ ∈ S for which d(s, s ′) < δ.

If s, s ′ ∈ S have the property that d(s, s ′) < δ then s, s ′ are within15 positions of each other in the sorted list Sy .

Divide and Conquer

Closest-Pair(P)1. Construct Px and Py ( O(n log n) time)

2. (p∗0 , p∗1) = Closest-Pair(Px ,Py )

Closest-Pair(Px ,Py )1. If |P| ≤ 3 compute the closest pairs by trying all and return...

2. Construct Qx ,Qy and Rx ,Ry ( O(n) time )

3. (q∗0 , q∗1) = Closest − Pair(Qx ,Qy )

4. (r∗0 , r∗1 ) = Closest − Pair(Rx ,Ry )

5. δ := min(d(q∗0 , q∗1), d(r∗0 , r

∗1 )).

6.x∗ = maximum x-coordinate of a point in set Q

7. L = (x , y) : x = x∗

8. S = points in P within distance δ of L

9. Construct Sy

Divide and Conquer

10. for each point s ∈ Sy , compute the distance from s to each ofnext 15 points in Sy .

11. Let s, s ′ be pair achieving minimum distance (O(n) time)

12. if d(s, s ′) < δ return (s, s ′)

13. if d(q∗0 , q∗1) < d(r∗0 , r

∗1 )) return (q∗0 , q

∗1)

14. else return (r∗0 , r∗1 )

Divide and Conquer

Matrix Multiplication

Given : We are given two n × n matrixes A,B of integers.

Goal : We want to compute A× B.

Direct approach: has O(n3) time complexity.

Divide and Conquer:

Divide and Conquer

C11 = A11 × B11 + A12 × B21

C12 = A11 × B12 + A12 × B22

C21 = A21 × B11 + A22 × B21

C22 = A21 × B12 + A22 × B22

In this case we have T (n) = 8T (n/2) +O(n2) = Θ(n3)How we can reduce the time complexity?

Divide and Conquer

C11 = A11 × B11 + A12 × B21

C12 = A11 × B12 + A12 × B22

C21 = A21 × B11 + A22 × B21

C22 = A21 × B12 + A22 × B22

In this case we have T (n) = 8T (n/2) +O(n2) = Θ(n3)How we can reduce the time complexity?

Divide and Conquer

Strassen’s Matrix Multiplication

In order to improve the algorithm we have to reduce the number ofmultiplication by introducing the new variables as follows:

P = (A11 + A22)× (B11 + B22)

Q = (A21 + A22)× B11

R = A11 × (B12 − B22)

S = A22 × (B21 − B11)

T = (A11 + A12)× B22

U = (A21 − A11)× (B11 + B12)

V = (A12 − A22)× (B21 + B22)

Now, we have:C11 = P + S − T + VC12 = R + TC21 = Q + SC22 = P + R − Q + U

So the T (n) can be expressed as:T (n) = 7T (n/2) + O(n2) = Θ(nlog2 7).

Divide and Conquer

Strassen’s Matrix Multiplication

In order to improve the algorithm we have to reduce the number ofmultiplication by introducing the new variables as follows:

P = (A11 + A22)× (B11 + B22)

Q = (A21 + A22)× B11

R = A11 × (B12 − B22)

S = A22 × (B21 − B11)

T = (A11 + A12)× B22

U = (A21 − A11)× (B11 + B12)

V = (A12 − A22)× (B21 + B22)

Now, we have:C11 = P + S − T + VC12 = R + TC21 = Q + SC22 = P + R − Q + U

So the T (n) can be expressed as:T (n) = 7T (n/2) + O(n2) = Θ(nlog2 7).

Divide and Conquer

Given an integer array of size n. Find the k-smallest number ?

Suppose n = 5(2r + 1) otherwise we add some zero to the end ofthe array.

Find the median for each group.

Divide and Conquer

Given an integer array of size n. Find the k-smallest number ?

Suppose n = 5(2r + 1) otherwise we add some zero to the end ofthe array.

Find the median for each group.

Divide and Conquer

Given an integer array of size n. Find the k-smallest number ?

Suppose n = 5(2r + 1) otherwise we add some zero to the end ofthe array.

Find the median for each group.

Divide and Conquer

Divide and Conquer

Divide and Conquer

Divide and Conquer