1

Programming Interest Grouphttp://www.comp.hkbu.edu.hk/~chxw/pig/index.htm

Tutorial Six

Divide and Conquer and Backtracking

2

Outline

Recursion Divide and Conquer Backtracking

Examples Constructing all subsets Constructing all permutations

Pruning Eight-Queens Problem

3

Recursion

A recursive function is one that calls itself. There must be a termination condition.

Example: Factorial function N! = Nx(N-1)x(N-2)x…x2x1 . 0! = 1

int factorial (unsigned int N) {

if (N == 0)

return 1;

return N*factorial(N-1);

}

4

Example: Euclid’s Algorithm

Find the greatest common divisors of two integers

int gcd (int m, int n) {

if (n == 0)

return m;

return gcd(n, m%n);

}

gcd(314159, 271828)

gcd(271828, 42331)

gcd(42331, 17842)

gcd(17842, 6647)

gcd(6647, 4458)

gcd(4458, 2099)

gcd(2099, 350)

gcd(350, 349)

gcd(349, 1)

gcd(1, 0)

return 1;

5

Recursive Functions

Advantage: Allow us to express complex algorithms in a compact form Recursive functions are the cornerstone of several

advanced techniques: backtracking, divide and conquer, dynamic programming

Disadvantage: We are nesting function calls Overhead of function calls Sometimes the program will fail because of the depth of

recursion is too large!

6

Fibonacci Number

Fn = Fn-1 + Fn-2; F0 = 0; F1 = 1 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, … How about writing a recursive function?

int fib( unsigned int n ) {

if (n == 0 || n == 1)

return n;

return fib(n-1) + fib(n-2);

}

Can you figure out the drawback of the left algorithm?

7

Divide and Conquer

Divide-and-conquer is a common and important strategy in problem-solving. Divide: to split the problem into two roughly equal sub-

problems, which are then solved recursively. Conquer: to patch together the two solutions of the sub-

problems, perform a small amount of additional work, and arrive at a solution for the whole problem

For divide-and-conquer approach, the sub-problems should be independent. Fibonacci number problem cannot be solved by divide-and-

conquer: Fn = Fn-1 + Fn-2

8

Divide and Conquer

Example: find the maximum among N items stored in an array a[0], …, a[N-1]

// a common and simple solution

for (t = a[0], i=1; i < N; i++)

if (a[i] > t)

t = a[i];

// divide and conquer

int max( int a[], int l, int r ) {

int m, u, v;

if (l == r) return a[l];

m = (l+r)/2;

u = max(a, l, m);

v = max(a, m+1, r);

if (u > v)

return u;

else

return v;

}

It’s used to show the concept. Its performance is not as

good as the left one.

9

Divide and Conquer

Example: Fast Exponentiation Calculate XN where N is an unsigned integer

int Exp( int x, unsigned int n ) {

int temp;

if (n == 0) return 1;

if (n == 1) return x;

if (n % 2 == 0) {

temp = Exp(x, n/2);

return temp * temp;

} else {

temp = Exp(x, n/2);

return temp * temp * x;

}

}

10

Divide and Conquer

Example: binary search Task: search an item in a sorted array

BinarySearch(int A[], int value, int low, int high) {

int mid;

if (high < low) return -1; // not found

mid = (low + high) / 2;

if (A[mid] > value)

return BinarySearch(A, value, low, mid-1);

else if (A[mid] < value)

return BinarySearch(A, value, mid+1, high);

else

return mid; // found

}

11

Divide and Conquer

Example: Mergesort A stable sorting algorithm with worst-case running

time of O(NlogN) Algorithm:

Divide the list into two sub-lists Sort each sub-list (recursion) Merge the two sorted sub-lists

12

Mergesortvoid msort(int A[], int temp[], int left, int right) {

int mid;

if (left < right) {

mid = (left + right) / 2;

msort(A, temp, left, mid);

msort(A, temp, mid+1, right);

merge(A, temp, left, mid+1, right);

}

}

void mergesort(int A[], int N) {

int *temp = malloc(N * sizeof(int));

if (temp != NULL) {

msort(A, temp, 0, N-1);

free(temp);

}

}

O(N)

13

Backtracking

Backtracking is a systematic method to iterate through all the possible configurations of a search space.

Traversal using right-hand rule

Ref: http://en.wikipedia.org/wiki/Maze

14

Backtracking

Given a problem, it may have a large number of different possible solutions, called “search space”, and your task is to find one correct solution or all correct solutions.

Brute force strategy: go through all possible solutions, and check them one by one

Today’s CPU (e.g., 3x109Hz) should be able to handle the problems with search space size of millions to billions.

15

Backtracking Size of search space is usually related to the

number of variables of the problem and the possible values of each variable

E.g., given 20 variables, each one can be either 0 or 1, then we have 220 different candidate solutions, which is around 1 million (106)

E.g., given 12 variables, each one can be 0, 1, 2, …, 9, then we have 1012 different candidate solutions! ---- brute force may not be a good idea for ACM contest

E.g., given 12 variables, which are the permutation of 1, 2, 3, …, 12, then we have 12! = 479,001,600 different candidate solutions. Brute force may work!

16

Backtracking How to iterate through all the possible configurations of a

search space? Recursion is the answer

Assume the problem has n variables, stored as a vector a = (a1, a2, …, an), and each variable ai is selected from a finite ordered set Si.

Backtracking:1. Process a partial solution (a1, a2, …, ak). If k == n, it is a complete

solution and we should handle it. Otherwise, goto Step 2.

2. Add one more variable ak+1, find the candidates for ak+1, i.e., Sk+1.

3. For each possible value of ak+1, process (a1, a2, …, ak+1) by recursion

17

Search Space

E.g., we have 4 variables, each could be 0 or 1 or 2

start

0 1 2a1

0 1 2a2

0 1 2a3

0 1 2a4

18

Backtracking A general programming structurebool finished = FALSE; /* to control when to stop */

backtrack(int a[], int k, data input ) {

int candidate[MAX]; /* candidates for next variable */

int ncandidates; /* number of candidates for next variable */

int i;

if ( is_a_solution(a, k, input) )

process_solution(a, k, input);

else {

k++;

construct_candidates(a, k, input, candidate, &ncandidates);

for (i = 0; i < ncandidates; i++) {

a[k] = candidate[i];

backtrack(a, k, input);

if (finished) return; /* we can terminate early by setting flag finished */

}

}

}

Some Comments is_a_solution(a, k, input)

Test whether the first k elements of vector a are a complete solution for the given problem

Argument input allows to pass general information into the routine, e.g., the size of a target solution. It could be ignored in some cases.

construct_candidates(a, k, input, c, ncandidates) Fill an array c with the complete set of possible candidates for

the kth position of a, given the contents of the first k-1 positions. The number of candidates returned is denoted by ncandidates

process_solution(a, k) Process a complete solution once it is constructed

19

20

Example 1: Constructing All Subsets

Given a set with n items, we have 2n subsets. How to output all the subsets?

E.g., the subsets of {1, 2, 3} include { 1 2 3 } { 1 2 } { 1 3 } { 1 } { 2 3 } { 2 } { 3 } { }

Introduce three variables, a1, a2, a3. Each of them can be either true or false.

ai is true means that i is in the subset.

21

Constructing All Subsetsis_a_solution(int a[], int k, int n) {

return (k == n);

}

process_solution(int a[], int k, int n) {

int i;

printf(“{ “);

for(i = 1; i <= k; i++) {

if (a[i] == TRUE)

printf(“ %d”, i);

printf(“ }\n”);

}

Reminder: In this example, the first variable is a[1]. a[0] is not used!

22

Constructing All Subsetsconstruct_candidates(int a[], int k, int input, int c[], int *n) {

c[0] = TRUE;

c[1] = FALSE;

*n = 2;

}

/* output all the subsets of {1, 2, 3} */

int main()

{

int n = 3;

int a[4];

backtrack(a, 0, n);

return 0;

}

Trace

23

backtrack( ( $, $, $), 0, 3 )

backtrack( ( T, $, $), 1, 3 ) backtrack( ( F, $, $), 1, 3 )

backtrack( ( T, T, $), 2, 3 ) backtrack( ( T, F, $), 2, 3 )

backtrack( ( T, T, T), 3, 3 ) {1 2 3}

backtrack( ( T, T, F), 3, 3 ) {1 2}

backtrack( ( T, F, T), 3, 3 ) {1 3}

backtrack( ( T, F, F), 3, 3 ) {1}

$: emptyT: TRUEF: FALSE

24

Stone PileTime Limit: 2.0 second Memory Limit: 16 MB

You have a number of stones with known weights W1…Wn. Write a program that will rearrange the stones into two piles such that weight difference between the piles is minimal.

Input contains the number of stones N (1 ≤ N ≤ 20) and weights of the stones W1…Wn (1 ≤ Wi ≤ 100000) delimited by white spaces.

Your program should output a number representing the minimal possible weight difference between stone piles.

Sample input55 8 13 27 14

output3

25

Example 2:Constructing All Permutations

Permutation of {1, 2, 3} include 123 132 213 231 312 321

Difference from the previous example:

When constructing the candidates for the next move, we must ensure that the ith element is distinct from all the elements before it.

26

Constructing All Permutationsis_a_solution(int a[], int k, int n) {

return (k == n);

}

process_solution(int a[], int k, int n) {

int i;

for(i = 1; i <= k; i++)

printf(“ %d”, a[i]);

printf(“\n”);

}

27

Constructing All Permutationsconstruct_candidates(int a[], int k, int input, int c[], int *n) {

int i;

int in_perm[NMAX];

for( i = 1; i < NMAX; i++) in_perm[i] = FALSE;

for( i = 1; i < k; i++) in_perm[ a[i] ] = TRUE;

*n = 0;

for (i = 1; i <= input; i++)

if (in_perm[i] == FALSE) { /* candidates must be */

c[*n] = i; /* different from existing */

*n = *n + 1; /* elements */

}

}

int main()

{

int n = 3;

int a[4];

backtrack(a, 0, n);

return 0;

}

28

Example 3:Eight-Queens Problem

The eight queens puzzle is the problem of putting eight chess queens on an 8×8 chessboard such that none of them is able to capture any other using the standard chess queen's moves.

A solution requires that no two queens share the same row, column, or diagonal.

How many solutions? http://en.wikipedia.org/wiki/Eight_queens_puzzle

29

Eight-Queens Problem

What is the search space? If a queen can be placed at any square, the search

space will be 648. Too large! There are several constrains, which help to reduce

the size of search space significantly. pruning

Consider the queen on the first column, it has 8 choices. Once it has been fixed, consider the queen on the second column, which has 7 choices. It’s easy to see that we have a total of 8! = 40320 cases only.

We haven’t used the diagonal constrain yet.

30

Eight-Queens Problemis_a_solution(int a[], int k, int n) {

return (k == n);

}

process_solution(int a[], int k, int n) {

solution_count++; // global count variable

}

int solution_count;

int main()

{

int n = 8; int a[9];

solution_count = 0;

backtrack(a, 0, n);

printf(“%d\n”, solution_count);

return 0;

}

31

Eight-Queens Problemconstruct_candidates(int a[], int k, int input, int c[], int *n) {

int i, j;

int legal_move;

*n = 0;

for (i = 1; i <= input; i++) {

legal_move = TRUE:

for (j = 1; j < k; j++) {

if ( abs(k-j) == abs(i-a[j]) ) legal_move == FALSE; /* diagonal threat */

if ( i == a[j] ) legal_move == FALSE; /* column threat */

}

if (legal_move == TRUE) {

c[*n] = i;

*n = *n +1;

}

}

}