Op#miza#on Problems, John Gu7ag

MIT Department of Electr ical Engineering and Computer Science

6.0002LECTURE2 1

§ Chapter13

Relevant Reading for Today’s Lecture

6.0002LECTURE2 2

The Pros and Cons of Greedy

§ Easytoimplement

§ Computa<onallyefficient

§ Butdoesnotalwaysyieldthebestsolu<on◦ Don’tevenknowhowgoodtheapproxima<onis

6.0002LECTURE2 3

Ques<on1

§ 1.Enumerateallpossiblecombina<onsofitems.

§ 2.Removeallofthecombina<onswhosetotalunitsexceedstheallowedweight.

§ 3.Fromtheremainingcombina<onschooseanyonewhosevalueisthelargest.

Brute Force Algorithm

6.0002LECTURE2 4

§ Thetreeisbuilttopdownstar<ngwiththeroot§ Thefirstelementisselectedfromthes<lltobeconsidereditems◦  Ifthereisroomforthatitemintheknapsack,anodeisconstructedthatreflectstheconsequenceofchoosingtotakethatitem.Byconven<on,wedrawthatastheleSchild

◦ Wealsoexploretheconsequencesofnottakingthatitem.Thisistherightchild

§ Theprocessisthenappliedrecursivelytonon-leafchildren§ Finally,choseanodewiththehighestvaluethatmeetsconstraints

Search Tree Implementa#on

6.0002LECTURE2 5

A Search Tree Enumerates Possibili#es

6.0002LECTURE2

6

Take Don’tTake

LeS-first,depth-firstenumera<on

Val=170Cal=766

Val=120Cal=766

Val=140Cal=508

Val=90Cal=145

Val=80Cal=612

Val=30Cal=258

Val=50Cal=354

Val=0Cal=0

6.0002LECTURE2 7

Image © source unknown. All rights reserved. This content is excluded from our CreativeCommons license. For more information, see https://ocw.mit.edu/help/faq-fair-use.

§ Timebasedonnumberofnodesgenerated

§ Numberoflevelsisnumberofitemstochoosefrom

§ Numberofnodesatleveliis2i

§ So,iftherearenitemsthenumberofnodesis◦ ∑𝑖=0↑𝑖=𝑛▒2↑𝑖  ◦  I.e.,O( 2↑𝑛+1 )

§ Anobviousop<miza<on:don’texplorepartsoftreethatviolateconstraint(e.g.,toomanycalories)◦ Doesn’tchangecomplexity

§ Doesthismeanthatbruteforceisneveruseful?◦ Let’sgiveitatry

Computa#onal Complexity

6.0002LECTURE2 8

Header for Decision Tree Implementa#on

6.0002LECTURE2 9

def maxVal(toConsider, avail): """Assumes toConsider a list of items, avail a weight Returns a tuple of the total value of a solution to 0/1 knapsack problem and the items of that solution""”

toConsider. Those items that nodes higher up in the tree (corresponding to earlier calls in the recursive call stack) have not yet considered

avail. The amount of space still available

Body of maxVal (without comments)

6.0002LECTURE2 10

if toConsider == [] or avail == 0: result = (0, ()) elif toConsider[0].getUnits() > avail: result = maxVal(toConsider[1:], avail) else: nextItem = toConsider[0] withVal, withToTake = maxVal(toConsider[1:], avail - nextItem.getUnits()) withVal += nextItem.getValue()

withoutVal, withoutToTake = maxVal(toConsider[1:], availf withVal > withoutVal: result = (withVal, withToTake + (nextItem,)) else: result = (withoutVal, withoutToTake) eturn result

oesnotactuallybuildsearchtree

) i r

DLocalvariableresultrecordsbestsolu<onfoundsofar

§ Withcaloriebudgetof750calories,choseanop<malsetoffoodsfromthemenu

Try on Example from Lecture 1

6.0002LECTURE2 11

Food wine beer pizza burger fries coke apple donut

Value 89 90 30 50 90 79 90 10

calories 123 154 258 354 365 150 95 195

§ Gaveusabeferanswer§ Finishedquickly§ But28isnotalargenumber◦ Weshouldlookatwhathappenswhenwehaveamoreextensivemenutochoosefrom

Search Tree Worked Great

6.0002LECTURE2 12

Code to Try Larger Examples

6.0002LECTURE2 13

import random

def buildLargeMenu(numItems, maxVal, maxCost): items = [] for i in range(numItems):

items.append(Food(str(i),random.randint(1, maxVal),random.randint(1, maxCost)))

return items

for numItems in (5,10,15,20,25,30,35,40,45,50,55,60): items = buildLargeMenu(numItems, 90, 250) testMaxVal(items, 750, False)

§ Intheory,yes§ Inprac<ce,no!§ Dynamicprogrammingtotherescue

Is It Hopeless?

6.0002LECTURE2 14

Some<mesanameisjustaname“The1950swerenotgoodyearsformathema<calresearch…IfeltIhadtodosomethingtoshieldWilsonandtheAirForcefromthefactthatIwasreallydoingmathema<cs...What<tle,whatname,couldIchoose?...It'simpossibletousetheworddynamicinapejora<vesense.Trythinkingofsomecombina<onthatwillpossiblygiveitapejora<vemeaning.It'simpossible.Thus,Ithoughtdynamicprogrammingwasagoodname.ItwassomethingnotevenaCongressmancouldobjectto.SoIuseditasanumbrellaformyac<vi<es.--RichardBellman

Dynamic Programming?

6.0002LECTURE2 15

Recursive Implementa#on of Fibonnaci

6.0002LECTURE2 16

def fib(n): if n == 0 or n == 1:

return 1 else:

return fib(n - 1) + fib(n - 2)

fib(120) = 8,670,007,398,507,948,658,051,921

Call Tree for Recursive Fibonnaci(6) = 13

6.0002LECTURE2 17

fib(6)

fib(5)

fib(4)

fib(3)

fib(2)

fib(1) fib(0)

fib(1)

fib(2)

fib(1) fib(0)

fib(3)

fib(2)

fib(1) fib(0)

fib(1)

fib(4)

fib(3)

fib(2)

fib(1) fib(0)

fib(1)

fib(2)

fib(1) fib(0)

§ Tradea<meforspace

§ Createatabletorecordwhatwe’vedone◦ Beforecompu<ngfib(x),checkifvalueoffib(x)alreadystoredinthetable◦ Ifso,lookitup◦ Ifnot,computeitandthenaddittotable

◦ Calledmemoiza<on

Clearly a Bad Idea to Repeat Work

6.0002LECTURE2 18

Using a Memo to Compute Fibonnaci

6.0002LECTURE2 19

def fastFib(n, memo = {}): """Assumes n is an int >= 0, memo used only by

recursive calls Returns Fibonacci of n"""

if n == 0 or n == 1: return 1

try: return memo[n]

except KeyError: result = fastFib(n-1, memo) +\

fastFib(n-2, memo) memo[n] = result return result

§ Op<malsubstructure:agloballyop<malsolu<oncanbefoundbycombiningop<malsolu<onstolocalsubproblems◦ Forx>1,fib(x)=fib(x-1)+fib(x–2)

§ Overlappingsubproblems:findinganop<malsolu<oninvolvessolvingthesameproblemmul<ple<mes◦ Computefib(x)ormany<mes

When Does It Work?

6.0002LECTURE2 20

§ Dothesecondi<onshold?

What About 0/1 Knapsack Problem?

6.0002LECTURE2 21

Ques<ons2and3

Search Tree

6.0002LECTURE2

22

Take Don’tTake

Val=170Cal=766

Val=120Cal=766

Val=140Cal=508

Val=90Cal=145

Val=80Cal=612

Val=30Cal=258

Val=50Cal=354

Val=0Cal=0

Op<malsubstructure?Overlappingsubproblems?

A Different Menu

6.0002LECTURE2

23

Take Don’tTake

Need Not Have Copies of Items

6.0002LECTURE2 24

Item Value Caloriesa 6 3b 7 3c 8 2d 9 5

§ Eachnode=<taken,leS,value,remainingcalories>

Search Tree

6.0002LECTURE2 25

§ Givenremainingweight,maximizevaluebychoosingamongremainingitems

§ Setofpreviouslychosenitems,orevenvalueofthatset,doesn’tmafer!

What Problem is Solved at Each Node?

6.0002LECTURE2 26

Overlapping Subproblems

6.0002LECTURE2 27

§ Addmemoasathirdargument◦ def fastMaxVal(toConsider, avail, memo = {}):

§ Keyofmemoisatuple◦ (itemsleStobeconsidered,availableweight)◦ ItemsleStobeconsideredrepresentedbylen(toConsider)

§ Firstthingbodyoffunc<ondoesischeckwhethertheop<malchoiceofitemsgiventhetheavailableweightisalreadyinthememo

§ Lastthingbodyoffunc<ondoesisupdatethememo

Modify maxVal to Use a Memo

6.0002LECTURE2 28

Performance

6.0002LECTURE2 29

len(items) 2**len(items) Numberofcalls

2 4 7

4 16 25

8 256 427

16 65,536 5,191

32 4,294,967,296 22,701

64 18,446,744,073,709 42,569,551,616

128 Big 83,319

256 ReallyBig 176,614

512 Ridiculouslybig 351,230

1024 Absolutelyhuge 703,802

§ Problemisexponen<al

§ Haveweoverturnedthelawsoftheuniverse?§ Isdynamicprogrammingamiracle?

§ No,butcomputa<onalcomplexitycanbesubtle

§ Running<meoffastMaxValisgovernedbynumberofdis<nctpairs,<toConsider, avail>◦ NumberofpossiblevaluesoftoConsiderboundedbylen(items)

◦ Possiblevaluesofavailabithardertocharacterize◦ Boundedbynumberofdis<nctsumsofweights

◦ Coveredinmoredetailinassignedreading

How Can This Be?

6.0002LECTURE2 30

§ Manyproblemsofprac<calimportancecanbeformulatedasop<miza<onproblems§ GreedyalgorithmsoSenprovideadequate(thoughnotnecessarilyop<mal)solu<ons§ Findinganop<malsolu<onisusuallyexponen<allyhard§ ButdynamicprogrammingoSenyieldsgoodperformanceforasubclassofop<miza<onproblems—thosewithop<malsubstructureandoverlappingsubproblems◦ Solu<onalwayscorrect◦ Fastundertherightcircumstances

Summary of Lectures 1-2

6.0002LECTURE2 31

The “Roll-over” Op#miza#on Problem

6.0002LECTURE2 32

Score=((60–(a+b+c+d+e))*F+a*ps1+b*ps2+c*ps3+d*ps4+e*ps5

Objec<ve:GivenvaluesforF,ps1,ps2,ps3,ps4,ps5Findvaluesfora,b,c,d,ethatmaximizescore

Constraints:a,b,c,d,eareeach10or0

a+b+c+d+e≥20

MIT OpenCourseWarehttps://ocw.mit.edu

6.0002 Introduction to Computational Thinking and Data ScienceFall 2016

For information about citing these materials or our Terms of Use, visit: https://ocw.mit.edu/terms.