PROBLEM-SOLVING TECHNIQUES Rocky K. C. Chang November 10, 2015 Objectives Understand and be able to use several approaches to solve problems: Forward or backward reasoning Greedy or not greedy Dynamic programming Divide and conquer approach A binary approach (Computer) Algorithms Mr. Webster said, an algorithm is a procedure for solving a mathematical problem in a finite number of steps that frequently involves repetition of an operation; Broadly: a step-by-step procedure for solving a problem or accomplishing some end especially by a computer. The number of steps may be finite (e.g., gcd) or infinite (e.g., the computation). What have you seen or will see? Involving numbers GCD computation The rabbit population problem Birthday cake cutting Permutation and combinatorial problems Basketball score Optimization problems X-to-Y problem Rock-moving game Coin changing problems (new) What have you seen or will see? Games Tower of Hanoi An integer-calling game A coin collection game (new) Involving a list of data Find the kth largest item in a list. (new) Correctness and performance An algorithm is correct if it always gives the expected solution to a problem. A proof for correctness A proof for incorrectness A problem but more than one solution Which algorithm is better and why? Why do we care? Forward and backward reasoning Forward reasoning: Start with what we know and advance toward the result to be obtained. Forward induction: Given that the smaller problem can be solved, what can we say about the solution to a larger problem? Backward reasoning: Start with the result to be obtained and advance toward what we know. Backward induction: Given that a larger problem can be solved, what can we say about the solution to a smaller problem? For example, X-to-Y problem (Backward) Start from Y and reason about the value before reaching Y. (Forward) We will see how it can be done. Basketball score (Backward) Start from the given score and consider all the possible states reaching this score. (Forward) It can be done Tower of Hanoi (Backward) Start from the last couple steps and go backward to the starting point of the game (Forward) It can be done. Both approaches can be based on a recursive definition but with an iterative implementation. Greedy or not greedy A greedy algorithm is one that always chooses the best option in each step. E.g., a forward-greedy approach to the X Y problem. Use 2 whenever possible. Not always the best solution, e.g., X = 3 and Y = 11. Greedy is not necessarily bad! A backward greedy approach to the X Y problem A base case: 2X > Y An inductive step partly based on a greedy approach An electrician problem An electrician connects a white dot and a black dot using a wire. Given N white dots and N black dots, how to connect them using the least amount of wires? Assume that the wire connecting two adjacent dots is 1. For example: Design a greedy algorithm for this problem. Does this algorithm always give the correct solution? When to be greedy? Usually involve in finding a max/min result in a problem, e.g., Minimum number of operations Minimum number of coins Shortest path from point A to point B. A few algorithms for graph problems are based on greedy approaches, e.g., Dijkstras shortest path algorithm Kruskals maximum spanning tree algorithm Dynamic programming Dynamic programming is a method for solving a complex problem by breaking it down into a collection of simpler subproblems, solving each just once, and storing their solutions. It can be used to solve optimization problems, e.g., the X Y problem and coin changing problem, and combinatorial problems, such as the basketball scores problem and the different combinations of the coins. Some shortest-path algorithms are also based on dynamic programming, such as the Warshalls algorithm. DP: Fibonacci numbers Use dynamic programming to find the Fibonacci numbers. Modify your code for Q1 of A7 by Using a list to store Fib(0), Fib(1), , Fib(n) Initializing Fib(0) = 0 and Fib(1) = 1, and Calculating Fib(i), i > 1. Use a time module to compute the total time for computing Fib(n) by adding the following codes: start_time = time.time() print("--- %s seconds ---" % (time.time() - start_time)) DP: The X Y problem Let M be a list containing the solutions to the problem of making X to X, X+1, X+2,..., Y. M[y] is the minimum no. of operations to make X equal to y, where y = X, X+1, X+2,..., Y. Therefore, the solution for this problem is given by M[Y]. The idea is that we can obtain M[Y] from the solutions to M[y], y < Y, by updating their optimal solutions. DP: The X Y problem The algorithm: Initialization: M[X] = 0, M[i] = Y, i = X+1,..., Y (just big enough) for i from X to Y-1 do M[i+1] min{M[i]+1, M[i+1]} if 2i