cs475 knapsack problem and dynamic programmingcs475/f16/lectures/... · travelling salesman problem...
TRANSCRIPT
9/27/16
1
cs475KnapsackProblemandDynamic
ProgrammingWimBohm,CS,CSU
sources:Cormen,Leiserson;Kleinberg,Tardos,VipinKumaret.al.
DynamicProgrammingApplications
Areas. Search Bioinformatics Controltheory Operationsresearch
Somefamousdynamicprogrammingalgorithms. Unixdiffforcomparingtwofiles. Knapsack Smith-Watermanforsequencealignment.
9/27/16
2
Fibonaccinumbers F(1)=F(2)=1,F(n)=F(n-1)+F(n-2)n>2
Simplerecursivesolution
deffib(n):
ifn<=2:return1
else:returnfib(n-1)+fib(n-2)
What is the size of the call tree?
5
4 3
3 2
2 1
2 1
exponential
Usingamemotabledeffib(n,table):
#pre:n>0,table[i]either0orcontainsfib(i)
ifn<=2:return1
iftable[n]>0:returntable[n]
result=fib(n-1,table)+fib(n-2,table)
table[n]=result
returnresult
Weuseamemotable,nevercomputingthesamevaluetwice.Howmanycallsnow?
9/27/16
3
Lookma,notabledeffib(n):
ifn<=2:return1
else
a=1;b=1,c=0
don-2times:c=a+b;a=b;b=c
returnc
5
Computethevalues"bottomup”Onlystoreneededvalues:theprevioustwoSameO(n)timecomplexity,constantspace
DynamicProgramming
• Characterizethestructureoftheproblem,ieshowhowalargerproblemcanbesolvedusingsolutionstosub-problems
• Recursivelydefinetheoptimum
• Computetheoptimumbottomup,storingvaluesofsubsolutions
• Constructtheoptimumfromthestoreddata
9/27/16
4
Optimalsubstructure
Dynamicprogrammingworkswhenaproblemhasoptimalsubstructure:wecanconstructtheoptimumofalargerproblemfromtheoptimaofa"smallset"ofsmallerproblems. small:polynomial
Notallproblemshaveoptimalsubstructure.
TravellingSalesmanProblem(TSP)?
KnapsackProblem
Givennobjectsanda"knapsack”ofcapacityW Itemihasaweightwi>0andvaluevi>0. Goal:fillknapsacksoastomaximizetotalvalue. IsthereaGreedysolution?
What’sGreedyagain? Whatwoulditdohere?
repeatedly add item with maximum vi / wi ratio … Does Greedy work?
Capacity M = 7, Number of objects n = 3 w = [5, 4, 3] v = [10, 7, 5] (ordered by vi / wi ratio)
9/27/16
5
RecursionforKnapsackProblem Notation:OPT(i,W)=optimalvalueofmaxweightsubsetthatusesitems1,…,iwithweightlimitW. Case1:itemiisnotincluded:
Takebestof{1,2,…,i-1}usingweightlimitW:OPT(i-1,W)
Case2:itemiwithweighwiandvalueviisincluded: onlypossibleifW>=wi newweightlimit=W–wi
Takebestof{1,2,…,i–1}usingweightlimitW-wiandaddvi:OPT(i-1,W-wi)+vi
OPT (i, W ) =0 if i = 0OPT (i−1, W ) if wi >Wmax OPT (i−1, W ), vi + OPT (i−1, W −wi ){ } otherwise
"
#$$
%$$
KnapsackProblem:Bottom-UpDynamicProgramming
Knapsack.Fillann-by-Warray.
Input: n, W, w1,…,wN, v1,…,vN for w = 0 to W M[0, w] = 0 for i = 1 to n for w = 0 to W if wi > w : M[i, w] = M[i-1, w] else : M[i, w] = max (M[i-1, w], vi + M[i-1, w-wi ]) return M[n, W]
9/27/16
6
KnapsackAlgorithm
n + 1
1
Value
18
22
28
1 Weight
5
6
6 2
7
Item 1
3
4
5
2
φ
{ 1, 2 }
{ 1, 2, 3 }
{ 1, 2, 3, 4 }
{ 1 }
{ 1, 2, 3, 4, 5 }
0
0
0
0
0
0
0
1
0
1
1
1
1
1
2
0
6
6
6
1
6
3
0
7
7
7
1
7
4
0
7
7
7
1
7
5
0
7
18
18
1
18
6
0
7
19
22
1
22
7
0
7
24
24
1
28
8
0
7
25
28
1
29
9
0
7
25
29
1
34
10
0
7
25
29
1
35
11
0
7
25
40
1
40
W + 1
W = 11
OPT: 40 How do we find the choice vector x, in other words the objects picked in the optimum solution?
Walk back through the table!!
KnapsackAlgorithm
n + 1
1
Value
18
22
28
1 Weight
5
6
6 2
7
Item 1
3
4
5
2
φ
{ 1, 2 }
{ 1, 2, 3 }
{ 1, 2, 3, 4 }
{ 1 }
{ 1, 2, 3, 4, 5 }
0
0
0
0
0
0
0
1
0
1
1
1
1
1
2
0
6
6
6
1
6
3
0
7
7
7
1
7
4
0
7
7
7
1
7
5
0
7
18
18
1
18
6
0
7
19
22
1
22
7
0
7
24
24
1
28
8
0
7
25
28
1
29
9
0
7
25
29
1
34
10
0
7
25
29
1
35
11
0
7
25
40
1
40
W + 1
W = 11
OPT: 40 n=5 Don’t take object 5
9/27/16
7
KnapsackAlgorithm
n + 1
1
Value
18
22
28
1 Weight
5
6
6 2
7
Item 1
3
4
5
2
φ
{ 1, 2 }
{ 1, 2, 3 }
{ 1, 2, 3, 4 }
{ 1 }
{ 1, 2, 3, 4, 5 }
0
0
0
0
0
0
0
1
0
1
1
1
1
1
2
0
6
6
6
1
6
3
0
7
7
7
1
7
4
0
7
7
7
1
7
5
0
7
18
18
1
18
6
0
7
19
22
1
22
7
0
7
24
24
1
28
8
0
7
25
28
1
29
9
0
7
25
29
1
34
10
0
7
25
29
1
35
11
0
7
25
40
1
40
W + 1
W = 11
OPT: 40 n=5 Don’t take object 5 n=4 Take object 4
KnapsackAlgorithm
n + 1
1
Value
18
22
28
1 Weight
5
6
6 2
7
Item 1
3
4
5
2
φ
{ 1, 2 }
{ 1, 2, 3 }
{ 1, 2, 3, 4 }
{ 1 }
{ 1, 2, 3, 4, 5 }
0
0
0
0
0
0
0
1
0
1
1
1
1
1
2
0
6
6
6
1
6
3
0
7
7
7
1
7
4
0
7
7
7
1
7
5
0
7
18
18
1
18
6
0
7
19
22
1
22
7
0
7
24
24
1
28
8
0
7
25
28
1
29
9
0
7
25
29
1
34
10
0
7
25
29
1
35
11
0
7
25
40
1
40
W + 1
W = 11
OPT: 40 n=5 Don’t take object 5 n=4 Take object 4 n=3 Take object 3 and now we cannot take anymore, so choice set is {3,4}
9/27/16
8
KnapsackProblem:RunningTime
Runningtime.Θ(nW). Notpolynomialininputsize! Wcanbeexponentialinn
"Pseudo-polynomial.”
Knapsackapproximation:thereexistsapoly-timealgorithmthatproducesafeasiblesolutionthathasvaluewithin0.01%ofoptimum
DIY:anotherexample
W=5461P=7894M=10
9/27/16
9
Example
W=5461P=7894M=10
cap
obj012345678910
{}00000000000
100000777777
Example
W=5461P=7894M=10
cap
obj012345678910
{}00000000000
100000777777
1:20000888881515
9/27/16
10
Example
W=5461P=7894M=10
cap
obj012345678910
{}00000000000
100000777777
1:20000888881515
1:30000889991517
Example
W=5461P=7894M=10
cap
obj012345678910
{}00000000000
100000777777
1:20000888881515
1:30000889991517
1:404448121213131519take4WHY?
9/27/16
11
Example
W=5461P=7894M=10
cap
obj012345678910
{}00000000000
100000777777
1:20000888881515
1:30000889991517not3WHY?
1:404448121213131519take4WHY?
Example
W=5461P=7894M=10
cap
obj012345678910
{}00000000000
100000777777
1:20000888881515take2WHY?
1:30000889991517not3WHY?
1:404448121213131519take4WHY?
9/27/16
12
Example
W=5461P=7894M=10
cap
obj012345678910
{}00000000000
100000777777take1WHY?
1:20000888881515take2WHY?
1:30000889991517not3WHY?
1:404448121213131519take4WHY?
ExampleW=5461P=7894M=10
cap
obj012345678910
{}00000000000
100000777777take1WHY?
1:20000888881515take2WHY?
1:30000889991517not3WHY?
1:404448121213131519take4WHY?
Solutionvector:1101,optimum:19