320201 algds spring 2016 lecture notesvisualization and computer graphics lab jacobs university...

CH08-320201: Algorithms and Data Structures 357

Visualization and Computer Graphics LabJacobs University

Reconstructing LCS• Trace backwards:

• Time complexity = O(m+n).



Fibonacci numbers revisitedRecall:• Recursive definition:

• Recursion tree of brute-force implementation:



Fibonacci numbers revisited

Dynamic programming solution:• Avoid re-computations of same terms.• Store results of subproblems in a table.• Thus, Fib(k) is computed exactly once for each k.• This basically leads to the previously discussed

bottom-up approach.• Computation time is T(n) = Θ(n).



4.2 Greedy Algorithms



Activity-selection problem

• Suppose we have a set S={a1,a2,…,an} of n activities.• The activities wish to use a resource, which can only

be used by one activity at a time.• Each activity ai has a start time si and a finish time fi,

where 0 ≤ si < fi < ∞.• Two activities ai and aj are compatible, if [si,fi) and

[sj,fj) are disjoint.• The activity-selection problem is to select a

maximum-size subset of mutually compatible activities.



Activity-selection problem

• Example:

• {a3,a9,a11} is a subset of mutually compatible activities.• {a1,a4,a8,a11} is a largest subset of mutually compatible

activities.• {a2,a4,a9,a11} is another largest subset of mutually

compatible activities.



Sorting

• We can apply a sorting algorithm to the finish times, which operates in O(n lg n) time.

• Then, we can assume that the activities are sorted, i.e.,



Greedy algorithm

• A greedy algorithm always makes the choice that looks best at the moment.

• I.e., it makes a locally optimal choice in the hope that it will lead to a globally optimal solution.



Greedy approach• After sorting, a1 has the earliest finish time f1.• A greedy approach starts with taking a1 as a locally

optimal choice.• Lemma:

The greedy choice of picking a1 as first choice is optimal.

• Proof:– Suppose A is a globally optimal solution for set S.– Let ak є A be the activity with earliest finish time fk in A.– If k=1, then a1 є A and we are done.– If k>1, then we can replace A by (A \ {ak}) U {a1}.– Since f1 ≤ fk, this is still an optimal solution.– Hence, we can always start with a1.



Greedy approach

• After the first step, we consider the subproblem S‘ = {ai є S: si ≥ f1}.

• We apply the same greedy strategy.• Lemma:

A \ {a1} is the optimal solution for S‘.• Proof:

– Let B be a solution for S‘ that is larger than A \ {a1}.– Then, B U {a1} would be solution for S that is larger than A.– Contradiction!



Greedy approach

• From the two lemmata, it follows by induction that the greedy approach delivers the globally optimal solution.



Greedy algorithm

Greedy-Selector (S)// Assume S = {a1,…,an} with activities sorted by fi.A := {a1}j := 1for i := 2 to n

if si ≥ fj

then A := A U {ai}j := i

return A



Greedy approach

• Time complexity:O(n lg n)

• Comparison to brute-force approach:– Brute-force approach would try all combinations, reject all

the combinations that have incompatibilities, and pick among the remaining ones one with maximum number of activities.

– Time complexity: O(2n)



Example



Greedy algorithms

• Just like dynamic programming, greedy algorithms build upon solving subproblems.

• However, greedy approaches make a locally optimal choice.

• There is no guarantee that this will lead to a globally optimal solution.

• Having found a global optimum requires a proof.



Knapsack problem



Problem: • Given some items, pack the knapsack to get the

maximum total value. • Each item has some weight and some value. • Total weight that we can carry is no more than some

fixed number W.• We must consider weights of items as well as their

values.

Item # Weight Value1 1 82 3 63 5 5

Knapsack problem



• Given a knapsack with maximum capacity W, and a set S consisting of n items

• Each item i has some weight wi and benefit value bi(assuming that all wi and W are integer values)

• How to pack the knapsack to achieve maximum total value of packed items?

• Mathematically:

Knapsack problem

∑∑∈∈

≤Ti

iTi

i Wwb subject to max



Brute-force approachAlgorithm:• Generate all 2n subsets• Discard all subsets whose sum of the weights exceed W

(not feasible)• Select the maximum total benefit of the remaining

(feasible) subsets

Time complexity: O(2n)



Brute-force approach

S = {(item1, 5, $70), (item2, 10, $90), (item3, 25, $140)} , W = 25

Subsets:1. {}2. {(item1, 5, $70)} Profit=$703. {(item2, 10, $90)} Profit=$904. {(item3, 25, $140)} Profit=$1405. {(item1, 5, $70), (item2, 10, $90)}. Profit=$1606. {(item2, 10, $90), (item3, 25, $140)} exceeds W7. {(item1, 5, $70), (item3, 25, $140)} exceeds W8. {(item1, 5, $70), (item2, 10, $90),(item3, 25, $140)} exceeds W



Greedy approach

• Use a greedy approach to be more efficient.• What would be the greedy choice?

– Maximum beneficial item– Minimum weight item– Maximum weight item– Maximum benefit per unit item

• Asymptotic time complexity:O(n lg n)



Greedy 1: Maximum beneficial item

S = {(item1, 5, $70), (item2, 10, $90), (item3, 25, $140)}, W = 25

5 lb

$7010 lb

$90

$140

W = 25lb

25 lb

item1 item2 item3 Knapsack GreedySolution

25 lb$140

OptimalSolution

10 lb

5 lb $70

10 lb$90

=$140 =$160



S = {(item1, 5, $150), (item2, 10, $60), (item3, 20, $140)}W = 30

5 lb$150 10 lb

$60

$140 W =30lb

20 lb


5 lb

5 lb

20 lb

$150

$140

5 lb

10 lb $60

$150OptimalSolution=$210 =$290

Greedy 2: Minimum weight item



S = {(item1, 5, $150), (item2, 10, $60), (item3, 20, $140)}W = 30

5 lb$150 10 lb

$60

$140 W =30lb

20 lb


5 lb

5 lb

20 lb

$150

$140

20 lb

10 lb $60

$140

OptimalSolution=$200 =$290

Greedy 3: Maximum weight item



S = {(item1, 5, $50), (item2, 20, $140), (item3, 10, $60)}W = 30

5 lb$50

W =30lb

5 lb

5 lb

20 lb

item1

$140

20 lb

item2 Knapsack GreedySolution

10 lb

20 lb

$50

$140 $140

$60

OptimalSolution

B/W 1: $10

B/W 2: $6

B/W: $7

10 lb

$60

item3 =$190 =$200

Greedy 4: Maximum benefit per unit item



Greedy approach

• As we had said, the locally optimal choice of a greedy approach does not necessary lead to a globally optimal one.

• For the knapsack problem, the greedy approach actually fails to produce a globally optimal solution.

• However, it produces an approximation, which sometimes is good enough.



• Let us try a dynamic-programming approach.• We need to carefully identify the subproblems• If items are labeled 1..n, then a subproblem would be to

find an optimal solution for Sk = {items labeled 1, 2,..., k}.

Dynamic-programming approach



For S4:Total weight: 14Maximum benefit: 20

w1 =2b1 =3

w2 =4b2 =5

w3 =5b3 =8

w4 =3b4 =4

wi bi

10

85

54

43

32

Weight Benefit

9

Item#

4

3

2

1

5

S4

S5w1 =2b1 =3

w2 =4b2 =5

w3 =5b3 =8

w5 =9b5 =10

For S5:Total weight: 20Maximum benefit: 26

Solution for S4 is not part of the solution for S5

Dynamic-programming approachMax weight: W = 20

320201 algds spring 2016 lecture notesvisualization and computer graphics lab jacobs university...

Documents