dynamic programming i · draw the recursion tree and obtain a complexity bound from the tree. prove...

Introduction Radboud University Nijmegen

Dynamic programming I

Alexandra Silva

[email protected]

http://www.cs.ru.nl/~alexandra

Institute for Computing and Information SciencesRadboud University Nijmegen

30th September 2014

Alexandra 30th September 2014 Lesson 4 1 / 24

Message of two weeks ago

• Recursive algorithms

• Intuitive but harder to analyse (recurrences)

• This week: dynamic programming.

Dynamic programming

• Simple, powerful, algorithm design technique

• Good for optimisation problems (min, max, shortest path, etc)

• Invented by Bellman.

Bellman (. . . ) explained that he invented the term dynamicprogramming to hide the fact that he was doing mathematicalresearch (. . . ) He settled on the term Dynamic programmingbecause it would be difficult to give a pejorative meaning andbecause it was something that not even a congressman wouldobject to.” [John Rust 2006]

• Next two lectures: basic principles of DP and different styles.

Dynamic programming

Fibonacci numbers

Typical exercise, recursive algorithm.

int fib (int n) {if (n==0 || n==1) return 1;

else return fib(n-1) + fib(n-2);

• Write a recurrence relation for this algorithm.

• Draw the recursion tree and obtain a complexity bound fromthe tree.

• Prove your complexity bound using the substitution method.

Fibonacci numbers

• T (n) = T (n − 1) + T (n − 2) + c

• Recursion tree (blackboard).

• Solution (one possibility!)

1 T is monotone: T (n) ≥ T (n − 2) + T (n − 2) = 2T (n − 2)2 How many times can we subtract 2 from n? n

3 T (n) ∈ O(2n/2).

Exponential, not great. . .

Fibonacci numbers

• T (n) = T (n − 1) + T (n − 2) + c

3 T (n) ∈ O(2n/2).

Fibonacci numbers

• T (n) = T (n − 1) + T (n − 2) + c

1 T is monotone: T (n) ≥ T (n − 2) + T (n − 2) = 2T (n − 2)2 How many times can we subtract 2 from n?

3 T (n) ∈ O(2n/2).

Fibonacci numbers

• T (n) = T (n − 1) + T (n − 2) + c

3 T (n) ∈ O(2n/2).

Fibonacci numbers

• T (n) = T (n − 1) + T (n − 2) + c

3 T (n) ∈ O(2n/2).

Fibonacci numbers

• T (n) = T (n − 1) + T (n − 2) + c

3 T (n) ∈ O(2n/2).

How can we do better?

• Ideas?

• Memoization (=remembering).

• Simple idea: every time we compute a fibonacci number, weput it in a dictionary.

• When we needs it, we check: is it there?

• Ideas?

Fibonacci numbers, memoized

memo = { }fib (n) =

if memo[n] is defined

return memo[n]

if (n==0 || n==1) return 1

else f = fib(n-1) + fib(n-2)

memo[n] = f

return f

• Blue part remains the same.

• Works for (almost) any recursive algorithm.

• Makes the algorithm more efficient.

Fibonacci numbers, memoized

memo = { }fib (n) =

if memo[n] is defined

return memo[n]

if (n==0 || n==1) return 1

else f = fib(n-1) + fib(n-2)

memo[n] = f

return f

• Blue part remains the same.

• Works for (almost) any recursive algorithm.

• Makes the algorithm more efficient.

Memoized Fibonacci, analysed

JJa/e v" <^rr^

^b^-o f^(^2.)

^bC^-2) fy\>(^ ^C^) P'^Cy^)\

4<JLT^ -KAJL O^WO>T^^ O^^JlTn^. ^ rc^t ^r^

<^«4,(x^ C AH '-..OM- ^l°w^ -)rYJV^\

. pb C^O ^^r^ rA^ H^ ^'^ ^ l+^ w^ ^ cM -g- . A^-^r>^/^ <<- ^^^Ji ^ ^ ^-cAo-x^... ^mo.^d <^|s a^- 6Cs) -c^\\^^ +^, ^w ^ ^t--^ c^^ ^ ^ Cf^-pb^J

^V.0 ,..-.,-^^-0>-^b(n)o Tk<, cno^ - reuuLF^tV^ 'uo^feL i^ cs^sfex^+- 6^3tf rc^l/u>^ -hvuj^ jCo .. Q C^^) -&i-»^^ I

î^ fy) n^ eêY) KL L>c^- (/vu <2x^\_Â; Q[?&}^)^ .+T>15I<1 ^C OYUîr' QSxrtCdULri

• Red part disappears in the memoized version (all rightsubtrees).

JJa/e v" <^rr^

^b^-o f^(^2.)

^bC^-2) fy\>(^ ^C^) P'^Cy^)\

4<JLT^ -KAJL O^WO>T^^ O^^JlTn^. ^ rc^t ^r^

<^«4,(x^ C AH '-..OM- ^l°w^ -)rYJV^\

. pb C^O ^^r^ rA^ H^ ^'^ ^ l+^ w^ ^ cM -g- . A^-^r>^/^ <<- ^^^Ji ^ ^ ^-cAo-x^... ^mo.^d <^|s a^- 6Cs) -c^\\^^ +^, ^w ^ ^t--^ c^^ ^ ^ Cf^-pb^J

^V.0 ,..-.,-^^-0>-^b(n)o Tk<, cno^ - reuuLF^tV^ 'uo^feL i^ cs^sfex^+- 6^3tf rc^l/u>^ -hvuj^ jCo .. Q C^^) -&i-»^^ I

î^ fy) n^ eêY) KL L>c^- (/vu <2x^\_Â; Q[?&}^)^ .+T>15I<1 ^C OYUîr' QSxrtCdULri

• Red part disappears in the memoized version (all rightsubtrees). Why?

• fib(k) recurses only the first time it is called for all k. (why?

afterwards it is stored in the dictionary)

• Memoized calls cost constant time.

• number of memoized calls is n :

fib(1). . . . , fib(n− 1), fib(n)

• The non-recursive work is constant.

• Running time is: linear!

(And this is not even the best one can do: O(lg n)! cf. exercises)

• fib(k) recurses only the first time it is called for all k. (why?afterwards it is stored in the dictionary)

• number of memoized calls is

fib(1). . . . , fib(n− 1), fib(n)

• Running time is:

linear!

fib(1). . . . , fib(n− 1), fib(n)

DP general idea

• memoize (remember);

• re-use solutions to sub-problems;

• use sub-problems to solve original problem.

• Challenge: Identify the sub-problems that allow you to solvethe original problem.

DP = recursion + memoization

DP = careful brute force

DP general idea

DP general idea, ctd.

• Running time:number of sub-problems x time per sub-problem

• For fib : n × c .

• DP = no recurrence solving (yay!).

• Note: memoized recursive calls do not count because after thefirst call they are for free (already stored in memory).

DP general idea, ctd.

• Running time:number of sub-problems x time per sub-problem

• For fib : n × c .

• DP = no recurrence solving (yay!).

• Note: memoized recursive calls do not count because after thefirst call they are for free (already stored in memory).

DP bottom-up algorithm

• Instead of thinking of a recursive algorithm top-down (cf.recursion tree last lecture). . .

• Do the reverse: work your way up!

• How does one compute Fibonacci numbers in primary school?

• Start: fib(0)=1, fib(1)=1, fib(2) = 1+1=2, . . .

fibonacci (n) :=

fib = { }for k in 0...n-1

if (k==0 || k==1) f=1

else f = fib[k-1] + fib[k-2]

fib[k] = f

return fib[n]

• Blue part remains the same as before (one small difference?).

• All transformations are automated: no thinking!

• In practice more efficient than memoized version. Why? nofunction calls.

fibonacci (n) :=

if (k==0 || k==1) f=1

fib[k] = f

return fib[n]

fibonacci (n) :=

if (k==0 || k==1) f=1

fib[k] = f

return fib[n]

• In practice more efficient than memoized version. Why?

nofunction calls.

fibonacci (n) :=

if (k==0 || k==1) f=1

fib[k] = f

return fib[n]

DP bottom-up algorithm – some notes

• When computing fib[k] one is sure fib[k-1] andfib[k-2] are there. Why?

• Ordering the sub-problems is important in the bottom upapproach.

• When computing fib[k] one is sure fib[k-1] andfib[k-2] are there. Why?

• Ordering the sub-problems is important in the bottom upapproach.

Bottom-up DP vs Memoized DP

• Computations are the same.

• Bottom up requires a topological sort of the subproblem DAG(directed acyclic graph).

In the fib case it is simple:

• Bottom-up perspective allows to save space. In the case offib: at each step, we only need two values. Linear time andconstant space.

• Bottom-up perspective: running time is obvious. In the caseof fib(n): loop runs n times (each step constant time).

• Bottom up requires a topological sort of the subproblem DAG(directed acyclic graph). In the fib case it is simple:

• Bottom-up perspective allows to save space.

In the case offib: at each step, we only need two values. Linear time andconstant space.

• Bottom-up perspective: running time is obvious.

In the caseof fib(n): loop runs n times (each step constant time).

DP example

Consider a row of n coins of values v(1) . . . v(n), where n is even.We play a game against an opponent by alternating turns. In eachturn, a player selects either the first or last coin from the row,removes it from the row permanently, and receives the value of thecoin. Determine the maximum possible amount of money we candefinitely win if we move first.

DP example-solution

Input row of n coins (even!) of values v1, . . . vn.

Goal maximise value of coins selected.

Sub-prob. V(i,j): max value we can definitely win if it is our turn andonly coins vi , . . . , vj remain.

Base case V(i,i) and V(i,i+1)

Result V(i,j) = max{

pick vi min{V (i + 1, j − 1),V (i + 2, j)}+ vi ,pick vj min{V (i , j − 2),V (i + 1, j − 1)}+ vj}

DP example-solution

Input row of n coins (even!) of values v1, . . . vn.

Goal maximise value of coins selected.

Sub-prob. V(i,j): max value we can definitely win if it is our turn andonly coins vi , . . . , vj remain.

Base case V(i,i) and V(i,i+1)

Result V(i,j) = max{

pick vi min{V (i + 1, j − 1),V (i + 2, j)}+ vi ,pick vj min{V (i , j − 2),V (i + 1, j − 1)}+ vj}

Shortest paths

• Naive recursive algorithm. How can one improve it?

• Idea to use: guessing!

• Don’t just try any guess, try all of them. (simple and smart?;-))

DP = recursion + memoization + guessing

DP = careful brute force(guessing is what needs care)

Shortest paths

Shortest path

Goal δ(s, v): shortest path between vertices s and v in a graph.

Idea There is some hypothetical shortest path. It starts from s.

What can we do in terms of guessing?

Option 1 Guess the first edge.

Shortest path

Goal δ(s, v): shortest path between vertices s and v in a graph.

Idea There is some hypothetical shortest path. It starts from s.

What can we do in terms of guessing?

Option 1 Guess the first edge.

Shortest path

Option 2 Guess the last edge.

• What is best?

• In option 2 the subproblem becomes δ(s, v ′): same shape, itstarts form s.

Shortest path

Option 2 Guess the last edge.

• What is best?

• In option 2 the subproblem becomes δ(s, v ′): same shape, itstarts form s.

Naive solution

C^,0-^ ^ Yw\^CU,U-)6 £

o.\\A^\ CJ^V^

^AO^ ^ ^u

Sc^,^ -\ ^c^>-o-\N\ _I

$^1, -\?roY»lt wv Cfl3+/vje< <

^ -^AAy-

£Ys,s) =.0 CY^ c^^ ^

TV^, ^ -^ "WArt w£ls"vv "^ ,-V ^OOcl7 A^. . ^^}^V j^ -e^onjLvz'b-'ciX Cc^cA<^ ! J ^

^ ^ ^ ^m^ . ^ ^ ^^ ^^ ^^b^ ^\^ ^^ . {-s^ ^ ^^^ c^ ^-^ ^

tovi. A^vAJfca^ &^ ^ ^ ^- ^/3-W ^ (2^^ .

^ .-v ysdL ^

Le^ ^s JLic^ .^&ji c^T7yi^-^Vi<M,

^^,^~)

Memoized solution

• Is the naive solution good?

no . . . terrible!

• It is exponential (check!).

• How do we memoize?

• We can build a V × V table (V are the nodes).

• It can be simpler: We fix s so we can just have a list storingδ(s,−).

• Is it good now?

Memoized solution

• Is the naive solution good? no . . . terrible!

• Is it good now?

Memoized solution

• Is it good now?

Memoized solution

• Is it good now?

Memoized solution

• It can be simpler:

We fix s so we can just have a list storingδ(s,−).

• Is it good now?

Memoized solution

• Is it good now?

Memoized solution-example

C^,0-^ ^ Yw\^CU,U-)6 £

o.\\A^\ CJ^V^

^AO^ ^ ^u

Sc^,^ -\ ^c^>-o-\N\ _I

^ -^AAy-

£Ys,s) =.0 CY^ c^^ ^

^ ^ ^ ^m^ . ^ ^ ^^ ^^ ^^b^ ^\^ ^^ . {-s^ ^ ^^^ c^ ^-^ ^

tovi. A^vAJfca^ &^ ^ ^ ^- ^/3-W ^ (2^^ .

^ .-v ysdL ^

^^,^~)

What happens for δ(s, v)?

?C<5)^0

q^luj) ^ 4 ^o 6L ^3^^-^ cXy^\\^.

<T> \a^L^A^ .\A

\A^ ^rT^

v^V ^o c^r^J- - .

CM^AL ^f^~J

J^^3)A-6-& C ^^^ a^d^C 0)(^^

G CV^)n/oVu C

C^,0-^ ^ Yw\^CU,U-)6 £

o.\\A^\ CJ^V^

^AO^ ^ ^u

Sc^,^ -\ ^c^>-o-\N\ _I

^ -^AAy-

£Ys,s) =.0 CY^ c^^ ^

^ ^ ^ ^m^ . ^ ^ ^^ ^^ ^^b^ ^\^ ^^ . {-s^ ^ ^^^ c^ ^-^ ^

tovi. A^vAJfca^ &^ ^ ^ ^- ^/3-W ^ (2^^ .

^ .-v ysdL ^

^^,^~)

What happens for δ(s, v)?

?C<5)^0

q^luj) ^ 4 ^o 6L ^3^^-^ cXy^\\^.

<T> \a^L^A^ .\A

\A^ ^rT^

v^V ^o c^r^J- - .

CM^AL ^f^~J

J^^3)A-6-& C ^^^ a^d^C 0)(^^

G CV^)n/oVu C

• Why is this a problem?

• Infinite algorithms in cyclic graphs! (oops, not so great...).

• In DAGs it executes in Θ(V + E ) (E are the edges). Why?

• Can we think of a solution that also works for cyclic graphs?

• Remember Manao’s problem: it is a matter of perspective!

• Whatever we do: acyclic ordering of subproblems. . .

• One possibility (there are different alternatives): ask insteadδk(s, v) shortest path from s to v using at most k edges.

• What is the solution of the original problem? What is thecomplexity of the memoized version? (Think!)

DP is a simple idea but requires care. . . more next time!

dynamic programming i · draw the recursion tree and obtain a complexity bound from the tree. prove...

Documents