rsh qam11 module02

Upload: primitivodelangel

Post on 02-Jun-2018

271 views

Category:

Documents


0 download

TRANSCRIPT

  • 8/10/2019 Rsh Qam11 Module02

    1/24

    M2

    1. Understand the overall approach of dynamic

    programming.

    2. Use dynamic programming to solve the shortest-

    route problem.

    3. Develop dynamic programming stages.

    After completing this module, students will be able to:

    2

    LEARNING OBJECTIVES

    Dynamic Programming

    MODULE

    Summary Glossary Key Equations Solved Problem Self-Test Discussion Questions and Problems Case

    Study: United Trucking Internet Case Study Bibliography

    4. Describe important dynamic programming

    terminology.

    5. Describe the use of dynamic programming in solvin

    knapsack problems.

    M2.1 Introduction

    M2.2 Shortest-Route Problem Solved Using Dynamic

    Programming

    M2.3 Dynamic Programming Terminology

    M2.4 Dynamic Programming Notation

    M2.5 Knapsack Problem

    MODULE OUTLINE

  • 8/10/2019 Rsh Qam11 Module02

    2/24

    M2-2 MODULE 2 DYNAMIC PROGRAMMING

    M2.1 Introduction

    Dynamic programming is a quantitative analysis technique that has been applied to large, com-

    plex problems that have sequences of decisions to be made. Dynamic programming divides

    problems into a number of decision stages; the outcome of a decision at one stage affects the de-

    cision at each of the next stages. The technique is useful in a large number of multiperiod busi-

    ness problems, such as smoothing production employment, allocating capital funds, allocating

    salespeople to marketing areas, and evaluating investment opportunities.

    Dynamic programming differs from linear programming in two ways. First, there is no al-gorithm (as in the simplex method) that can be programmed to solve all problems. Dynamic pro-

    gramming is, instead, a technique that allows us to break up difficult problems into a sequence

    of easier subproblems, which are then evaluated by stages. Second, linear programming is a

    method that gives single-stage (one time period) solutions. Dynamic programming has the

    power to determine the optimal solution over a one-year time horizon by breaking the problem

    into 12 smaller one-month horizon problems and to solve each of these optimally. Hence, it uses

    a multistage approach.

    Solving problems with dynamic programming involves four steps:

    Four Steps of Dynamic Programming

    1. Divide the original problem into subproblems called stages.2. Solve the last stage of the problem for all possible conditions or states.

    3. Working backward from the last stage, solve each intermediate stage. This is done by

    determining optimal policies from that stage to the end of the problem (last stage).

    4. Obtain the optimal solution for the original problem by solving all stages sequentially.

    In this module we show how to solve two types of dynamic programming problems: net-

    work and nonnetwork. The shortest-route problem is a network problem that can be solved by

    dynamic programming. The knapsack problem is an example of a nonnetwork problem that can

    be solved using dynamic programming.

    The second step is to solve the

    last stagestage 1.

    M2.2 Shortest-Route Problem Solved Using Dynamic Programming

    George Yates is about to make a trip from Rice, Georgia (1), to Dixieville, Georgia (7). George

    would like to find the shortest route. Unfortunately, there are a number of small towns between

    Rice and Dixieville. His road map is shown in Figure M2.1.

    The circles on the map, called nodes, represent cities such as Rice, Dixieville, Brown, and

    so on. The arrows, called arcs, represent highways between the cities. The distance in miles is

    indicated along each arc. This problem can, of course, be solved by inspection. But seeing how

    dynamic programming can be used on this simple problem will teach you how to solve larger

    and more complex problems.

    Step 1: The first step is to divide the problem into subproblems or stages. Figure M2.2 reveals

    the stages of this problem. In dynamic programming, we usually start with the last part of the

    problem, stage 1, and work backward to the beginning of the problem or network, which is stage

    3 in this problem. Table M2.1 summarizes the arcs and arc distances for each stage.

    Step 2: We next solve stage 1, the last part of the network. Usually, this is trivial. We find the

    shortest path to the end of the network, node 7 in this problem. At stage 1, the shortest paths,

    The first step is to divide the

    problem into subproblems or

    stages.

    Dynamic programming

    breaks a difficult problem

    into subproblems.

  • 8/10/2019 Rsh Qam11 Module02

    3/24

    M2.2 SHORTEST-ROUTE PROBLEM SOLVED USING DYNAMIC PROGRAMMING M2

    1 3

    2

    4

    6

    5

    7

    Stage 3

    A Branch

    Stage 2 Stage 1

    A Node

    45

    2

    10

    12

    6

    4 10

    2

    14

    FIGURE M2.2

    Three Stages for theGeorge Yates Problem

    10 Miles

    10 Miles

    12Mile

    s

    4Mile

    sBrown

    2Mile

    s

    14Miles

    5 Miles

    4Mile

    s

    2Miles

    Lake City

    Hope

    Athens

    Georgetown

    Dixieville

    4

    1 3

    2

    5

    6

    7

    6Miles

    Rice

    FIGURE M2.1

    Highway Map betweenRice and Dixieville

    STAGE ARC ARC DISTANCE

    1 57 14

    67 2

    2 45 10

    35 12

    36 6

    25 4

    26 10

    3 14 4

    13 5

    12 2

    TABLE M2.1

    Distance Along Each Arc

  • 8/10/2019 Rsh Qam11 Module02

    4/24

    M2-4 MODULE 2 DYNAMIC PROGRAMMING

    45

    2

    10

    12

    6

    410

    2

    14

    Minimum Distanceto Node 7 fromNode 6

    Minimum Distanceto Node 7 fromNode 5

    1 3

    2

    4

    6

    5

    7

    2

    14

    FIGURE M2.3

    Solution for the One-Stage Problem

    45

    2

    10Minimum Distanceto Node 7 fromNode 3

    12

    6

    4 10

    2

    14

    Minimum Distanceto Node 7 fromNode 4

    Minimum Distanceto Node 7 fromNode 2

    1 3

    2

    4

    6

    5

    7

    8

    12

    14

    2

    24

    FIGURE M2.4Solution for the Two-Stage Problem

    Step 3: Moving backward, we now solve for stages 2 and 3. At stage 2 we will use Figure M2.4.Step 3 involves moving backwardsolving intermediate stages.

    from node 5 to node 6, to node 7 are the only paths. Also note in Figure M2.3 that the minimum

    distances are enclosed in boxes by the entering nodes to stage 1, node 5 and node 6. The objec-

    tive is to find the shortest distance to node 7. The following table summarizes this procedure for

    stage 1. As mentioned previously, the shortest distance is the only distance at stage 1.

    STAGE 1

    BEGINNING SHORTEST DISTANCE ARCS ALONGNODE TO NODE 7 THIS PATH

    5 14 57

    6 2 67

  • 8/10/2019 Rsh Qam11 Module02

    5/24

    M2.2 SHORTEST-ROUTE PROBLEM SOLVED USING DYNAMIC PROGRAMMING M2

    1 3

    2

    4

    6

    5

    7

    45

    2

    10

    8

    12

    6

    4 10

    2

    14

    24

    Minimum Distanceto Node 7 fromNode 1

    12

    14

    2

    13

    FIGURE M2.5

    Solution for the Three-Stage Problem

    If we are at node 4, the shortest and only route to node 7 is arcs 45 and 57. At node 3, t

    shortest route is arcs 36 and 67 with a total minimum distance of 8 miles. If we are at node

    the shortest route is arcs 26 and 67 with a minimum total distance of 12 miles. This inform

    tion is summarized in the stage 2 table:

    STAGE 2

    BEGINNING SHORTEST DISTANCE ARCS ALONG

    NODE TO NODE 7 THIS PATH

    4 24 45

    57

    3 8 36

    67

    2 12 26

    67

    The solution to stage 3 can be completed using the following table and the network

    Figure M2.5:

    STAGE 3

    BEGINNING SHORTEST DISTANCE ARCS ALONGNODE TO NODE 7 THIS PATH

    1 13 13

    36

    67

  • 8/10/2019 Rsh Qam11 Module02

    6/24

    M2-6 MODULE 2 DYNAMIC PROGRAMMING

    M2.3 Dynamic Programming Terminology

    Regardless of the type or size of a dynamic programming problem, there are some important

    terms and concepts that are inherent in every problem. Some of the more important of them

    follow:

    1. Stage: a period or a logical subproblem.

    2. State variables: possible beginning situations or conditions of a stage. These have also

    been called the input variables.

    3. Decision variables: alternatives or possible decisions that exist at each stage.

    4. Decision criterion: a statement concerning the objective of the problem.

    5. Optimal policy: a set of decision rules, developed as a result of the decision criteria, that

    gives optimal decisions for any entering condition at any stage.

    6. Transformation: normally, an algebraic statement that reveals the relationship betweenstages.

    In the shortest-route problem, the following transformation can be given:

    This relationship shows how we are able to go from one stage to the next in solving for the

    optimal solution to the shortest-route problem. In more complex problems, we can use symbols

    to show the relationship between stages.

    State variables, decision variables, the decision criterion, and the optimal policy can be de-

    termined for any stage of a dynamic programming problem. This is done here for stage 2 of the

    George Yates shortest-route problem.

    Distancefromthe

    beginningofagiven

    stagetothelastnode

    =

    Distancefromthebeginning

    ofthepreviousstage

    tothelastnode

    +

    Distancefromthe

    givenstagetothe

    previousstage

    The fourth and final step is to

    find the optimal solution after all

    stages have been solved.

    Step 4: To obtain the optimal solution at any stage, all we consider are the arcs to the next stage

    and the optimal solution at the next stage. For stage 3, we only have to consider the three arcs to

    stage 2 (12, 13, and 14) and the optimal policies at stage 2, given in a previous table. This is

    how we arrived at the preceding solution. When the procedure is understood, we can perform all

    the calculations on one network. You may want to study the relationship between the networks

    and the tables because more complex problems are usually solved by using tables only.

    Dynamic Programming in Nursery

    Production Decisions

    Managing a nursery that produces ornamental plants is diffi-cult. In most cases, ornamental plants increase in value with in-

    creased growth. This value-added growth makes it difficult to

    determine when to harvest the plants and place them on the

    market. When plants are marketed earlier, revenues are gener-

    ated earlier and the costs associated with plant growth are mini-

    mized. On the other hand, delaying the harvesting of the

    ornamental plants usually results in higher prices. But are the ad-

    ditional months of growth and costs worth the delay?

    In this case, dynamic programming was used to determine

    the optimal growth stages for ornamental plants. Each stage was

    associated with a possible growth level. The state variables

    included acres of production of ornamental plants and carryover

    plants from previous growing seasons. The objective of the

    dynamic programming problem was to maximize the after-tax

    cash flow. The taxes included self-employment, federal income,

    earned income credit, and state income taxes.

    The solution was to produce one- and three-gallon containers

    of ornamental plants. The one-gallon containers are sold in the fall

    and carried over for spring sales. Any one-gallon containers not

    sold in the spring are combined into three-gallon container prod-

    ucts for sale during the next season. Using dynamic programming

    helps to determine when to harvest to increase after-tax cash flow.

    Source: Jeffrey Stokes, et al. Optimal Marketing of Nursery Crops from

    Container-Based Production Systems, American Journal of Agricultural

    Economics (February 1997): 235246.

    IN ACTION

  • 8/10/2019 Rsh Qam11 Module02

    7/24

    M2.3 DYNAMIC PROGRAMMING TERMINOLOGY M2

    1. State variables for stage 2 are the entering nodes, which are

    (a) node 2

    (b) node 3

    (c) node 4

    2. Decision variables for stage 2 are the following arcs or routes:

    (a) 45

    (b) 35

    (c) 36

    (d) 25(e) 26

    3. The decision criterion is the minimization of the total distances traveled.

    4. The optimal policy for any beginning condition is shown in Figure M2.6 and the followin

    table:

    1 3

    2

    4

    6

    5

    7

    10

    8

    12

    6

    4 10

    24

    12

    14

    2

    Statevariablesare theenteringnodes.

    The optimal policy is the arc,for any entering node, thatwill minimize total distanceto the destination at thisstage.

    Decisionvariablesare allthe arcs.

    Stage 2

    FIGURE M2.6

    Stage 2 from theShortest-Route Problem

    GIVEN THIS ENTERING THIS ARC WILL MINIMIZECONDITION TOTAL DISTANCE TO NODE 7

    2 26

    3 36

    4 45

  • 8/10/2019 Rsh Qam11 Module02

    8/24

    M2-8 MODULE 2 DYNAMIC PROGRAMMING

    An input, decision, output, and

    return are specified for each

    stage.

    The input to one stage is the

    output from another stage.

    M2.4 Dynamic Programming Notation

    In addition to dynamic programming terminology, we can also use mathematical notation to de-

    scribe any dynamic programming problem. This helps us to set up and solve the problem. Con-

    sider stage 2 in the George Yates dynamic programming problem first discussed in Section

    M2.2. This stage can be represented by the diagram shown in Figure M2.7 (as could any given

    stage of a given dynamic programming problem).

    As you can see, for every stage, we have an input, decision, output, and return. Look again

    at stage 2 for the George Yates problem in Figure M2.6. The input to this stage is which con-

    sists of nodes 2, 3, and 4. The decision at stage 2, or choosing which arc will lead to stage 1, is

    represented by The possible arcs or decisions are 45, 35, 36, 25, and 26. The output to

    stage 2 becomes the input to stage 1. The output from stage 2 is The possible outputs from

    stage 2 are the exiting nodes, nodes 5 and 6. Finally, each stage has a return. For stage 2, the re-

    turn is represented by In our shortest-route problem, the return is the distance along the arcs

    in stage 2. These distances are 10 miles for arc 45, 12 miles for arc 35, 6 miles for arc 36, 4

    miles for arc 25, and 10 miles for arc 26. The same notation applies for the other stages and

    can be used at any stage. In general, we use the following notation for these important concepts:

    (M2-1)

    (M2-2)

    (M2-3)

    You should also note that the input to one stage is also the output from another stage. For

    example, the input to stage 2, is also the output from stage 3 (see Figure M2.7). This leads us

    to the following equation:

    (M2-4)sn-1 = Outputfromstagen

    s2,

    rn = Returnatstagen

    dn = Decisionatstagen

    sn = Inputtostagen

    r2.

    s1.

    d2.

    s2,

    Figure M2.6 may also be helpful in understanding some of the terminology used in the dis-

    cussion of dynamic programming.

    Stage 2

    Decisiond2

    Output

    s1

    Return

    r2

    Input

    s2

    FIGURE M2.7

    Input, Decision, Output,and Return for Stage 2 inGeorge Yatess Problem

  • 8/10/2019 Rsh Qam11 Module02

    9/24

    M2.5 KNAPSACK PROBLEM M2

    A transformation function

    allows us to go from one stage

    to another.

    The total return function

    allows us to keep track of

    profits and costs.

    The objective of this problem is

    to maximize profits.

    M2.5 Knapsack Problem

    The knapsack problem involves the maximization or the minimization of a value, such as pro

    its or costs. As in a linear programming problem, there are restrictions in a knapsack proble

    Imagine a knapsack or pouch that can hold only a certain weight or volume. We can place d

    ferent types of items in the knapsack. Our objective is to place items in the knapsack to ma

    mize total value without breaking the knapsack because of too much weight or a simil

    restriction.

    Types of Knapsack ProblemsMany kinds of problems can be classified as knapsack problems. Choosing items to place in t

    cargo compartment of an airplane and selecting which payloads to put on the next NASA Spa

    Shuttle are examples. The restriction can be volume, weight, or both. Some scheduling pro

    lems are also knapsack problems. For example, we may want to determine which jobs to com

    plete in the next two weeks. The two-week period is the knapsack, and we want to load it wi

    jobs in such a way as to maximize profits or minimize costs. The restriction is the number

    days or hours during the two-week period.

    Rollers Air Transport Service Problem

    Rob Roller owns and operates Rollers Air Transport Service, which ships cargo by plane

    most large cities in the United States and Canada. The remaining capacity for one of t

    flights from Seattle to Vancouver is 10 tons. There are four different items that Rob can shbetween Seattle and Vancouver. Each item has a weight in tons, a net profit in thousands

    dollars, and a total number of that item that is available for shipping. This information is pr

    sented in Table M2.2.

    PROBLEM SETUP Rollers Air Transport problem is an ideal problem to solve using dynam

    programming. Stage 4 will be item 1, stage 3 will be item 2, stage 2 will be item 3, and sta

    The final concept is transformation. The transformation function allows us to go from o

    stage to another. The transformation function for stage 2, converts the input to stage 2, a

    the decision made at stage 2, to the output from stage 2, Because the transformation fun

    tion depends on the input and decision at any stage, it can be represented as In ge

    eral, the transformation function can be represented as follows:

    (M2

    The following general formula allows us to go from one stage to another using the transform

    tion function:

    (M2

    Although this equation may seem complex, it is really just a mathematical statement of the f

    that the output from a stage is a function of the input to the stage and any decisions made at th

    stage. In the George Yates shortest-route problem, the transformation function consists o

    number of tables. These tables show how we could progress from one stage to another in ord

    to solve the problem. For more complex problems, we need to use dynamic programming no

    tion instead of tables.

    Another useful quantity is the total return at any stage. The total return allows us to ke

    track of the total profit or costs at each stage as we solve the dynamic programming problem

    can be given as follows:

    (M2fn = Totalreturnatstagen

    sn-1 = tn1sn, dn2

    tn = Transformationfunctionatstagen

    t2(s2,d2).

    s1.d2,

    s2,t2,

  • 8/10/2019 Rsh Qam11 Module02

    10/24

    M2-10 MODULE 2 DYNAMIC PROGRAMMING

    TABLE M2.2

    Items to Be Shipped

    Defining the ProblemThe Southern Company, with service areas in Georgia, Alabama, Mississippi, and Florida, is a major

    provider of electric service, with about 240 generating units. In recent years, fuel costs have increased

    faster than other costs. Annual fuel costs are about $2.5 billion, representing about one-third of total ex-

    penses for the Southern Company. The problem for the Southern Company is to reduce total fuel costs.

    Developing a Model

    To deal with this fuel cost problem, the company developed a state-of-the-art dynamic programming

    model. The dynamic programming model is embedded in the Wescouger optimization program, which is

    a computer program used to control electric generating units and reduce fuel costs through better utiliza-

    tion of existing equipment.

    Acquiring Input DataData were collected on past and projected electric usage. In addition, daily load/generation data were an-

    alyzed. Load/generation charts were used to investigate the fuel requirements for coal, nuclear, hydroelec-

    tric, and gas/oil.

    Developing a SolutionThe solution of the dynamic programming model provides both short-term planning guidelines and long-

    term fuel usage for the various generating units. Optimal maintenance schedules for generating units are

    obtained using Wescouger.

    Testing the Solution

    To test the accuracy of the Wescouger optimization program, Southern used a real-time economic dis-

    patch program. The results were a very close match. In addition, the company put the solution through an

    acid test, in which seasoned operators compared the results against their intuitive judgment. Again, the

    results were consistent.

    Analyzing the Results

    The results were analyzed in terms of their impact on the use of various fuels, the usage of various gener-

    ating units, and maintenance schedules for generating units. Analyzing the results also revealed other

    needs. This resulted in a full-color screen editing routine, auxiliary programs to automate data input, and

    software to generate postanalysis reports.

    Implementing the ResultsThe Southern Company implemented the dynamic programming solution. Over a seven-year period, the

    results saved the company over $140 million.

    Source: S. Erwin, et al. Using an Optimization Software to Lower Overall Electric Production Costs for Southern Company,Interfaces 21, 1 (JanuaryFebruary 1991): 2741.

    MODELING IN THE REAL WORLDReducing Electric Production Costs

    Using Dynamic Programming

    Defining

    the Problem

    Developing

    a Model

    Acquiring

    Input Data

    Developing

    a Solution

    Testing the

    Solution

    Analyzing

    the Results

    Implementing

    the Results

    ITEM WEIGHT PROFIT/UNIT NUMBER AVAILABLE

    1 1 $3 6

    2 4 9 1

    3 3 8 2

    4 2 5 2

  • 8/10/2019 Rsh Qam11 Module02

    11/24

    M2.5 KNAPSACK PROBLEM M2-

    s1Stage 2(Item 3)

    Stage 3(Item 2)

    d4

    s3

    r4

    s4 Stage 4(Item 1)

    s0Stage 1(Item 4)

    d3

    r3

    d2

    r2

    d1

    r1

    Decisions

    Returns

    s2

    FIGURE M2.8

    Rollers Air TransportService Problem

    1 will be item 4. This is shown in Table M2.3. During the solution, we will be using sta

    numbers.

    Rollers Air Transport problem can be represented graphically (see Figure M2.8). As y

    can see, each item is represented by a stage. Look at stage 4, which is item 1. The total weig

    that can be used is represented by This amount is 10 because we havent assigned any un

    to be shipped at this time. The decision at this stage is (the number of units of item 1 we w

    ship). If is 1, for example, we will be shipping 1 unit of item 1. Also note that is the retu

    or profit at stage 4 (item 1). If we ship 1 unit of item 1, the profit will be $3 (see Table M2.2).

    As mentioned previously, the decision variable represents the number of units of ea

    item (stage) that can be shipped. Looking back at the original problem, we see that the proble

    is constrained by the number of units. This is summarized in the following table:

    dn,

    r4d4

    d4

    s4.

    TABLE M2.3

    Relationship BetweenItems and Stages

    STAGE MAXIMUM VALUE OF DECISION

    4 6

    3 1

    2 2

    1 2

    THE TRANSFORMATION FUNCTIONS Next, we need to look at the transformation function. T

    general transformation function for knapsack problems follows:

    Note that and are coefficients for the problem and that represents the decision

    stage n. This is the number of units to ship at stage sn.

    dncnan,bn,

    sn-1 = (an * sn) + (bn * dn) + cn

    ITEM STAGE

    1 4

    2 3

    3 2

    4 1

  • 8/10/2019 Rsh Qam11 Module02

    12/24

    M2-12 MODULE 2 DYNAMIC PROGRAMMING

    The following chart shows the transformation coefficients for Rob Rollers transport problem:

    COEFFICIENTS OFTRANSITION FUNCTION

    STAGE an

    bn

    cn

    4 1 1 0

    3 1 4 02 1 3 0

    1 1 2 0-

    -

    -

    -

    First, note that is 10, the total weight that can be shipped. Because represents the first item,

    all 10 tons can be utilized. The transformation equations for the four stages are as follows:

    (a)

    (b)

    (c)

    (d)

    Consider stage 3. Equation b reveals that the number of tons still available after stage 3, is

    equal to the number of tons available before stage 3, minus the number of tons shipped at

    stage In this equation, means that each item at stage 3 weighs 4 tons.

    THE RETURN FUNCTIONS Next, we will look at the return function for each stage. This is the

    general form for the return function:

    Note that and are the coefficients for the return function. Using this general form of

    the return function, we can put the return function values in the following table:

    cnan,bn,

    rn = 1an * sn2 + 1bn * dn2 + cn

    4d33,4d3.

    s3,

    s2,

    s0 = s1 - 2d1 stage1

    s1 = s2 - 3d2 stage2

    s2 = s3 - 4d3 stage3

    s3 = s4 - 1d4 stage4

    s4s4

    COEFFICIENTS OF

    DECISIONS RETURN FUNCTION

    STAGE LOWER UPPER an

    bn

    cn

    4 0 dn

    6 0 3 0

    3 0 dn

    1 0 9 0

    2 0 dn

    2 0 8 0

    1 0 dn

    2 0 5 0

  • 8/10/2019 Rsh Qam11 Module02

    13/24

    M2.5 KNAPSACK PROBLEM M2-

    The lower value for each decision is zero, and the upper value is the total number availab

    The coefficient is the profit per item shipped. The actual return functions are:

    STAGE-BY-STAGE SOLUTION As you would expect, the return at any stage is equal to t

    profit per unit at that stage multiplied by the number of units shipped at that stage, With t

    information, we can solve Rollers Air Transport problem, starting with stage 1 (item 4). T

    following table shows the solution for the first stage. You may wish to refer to Figure M2.8

    this discussion.

    dn.

    rn,

    r1 = 5d1

    r2 = 8d2

    r3 = 9d3

    r4 = 3d4

    bn

    Because we dont know how many tons will be available at stage 1, we must consider

    possibilities. Thus, the number of tons available at stage 1, can vary from 1 to 10. This is se

    in the first column of numbers for stage 1. The number of units that we ship at stage 1, c

    vary from 0 to 2. We cant go over 2 because the number available is only 2. For any decisi

    we compute the return at stage 1, by multiplying the number of items shipped by 5, the pro

    per item. The profit at this stage will be 0, 5, or 10, depending on whether 0, 1, or 2 items a

    shipped. Note that the total return at this stage, is the same as because this is the only sta

    we are considering so far. Also note that the total return before stage 1, is 0 because thisthe beginning of the solution and we are shipping nothing at this point.

    f0,

    r1f1,

    r1,

    d1,

    s1,

    We consider all possibilities.

    is the total return at the first

    stage. The total return before the

    first stage is f0.

    f1

    STAGE 1

    S1

    d1

    r1

    s0

    f0

    f1

    0 0 0 0 0 0

    1 0 0 0 0 0

    2 0 0 0 0 01 5 0 0 5

    3 0 0 0 0 0

    1 5 0 0 5

    4 0 0 0 0 0

    1 5 0 0 5

    2 10 0 0 10

    5 0 0 0 0 0

    1 5 0 0 5

    2 10 0 0 10

    6 0 0 0 0 01 5 0 0 5

    2 10 0 0 10

    STAGE 1

    S1

    d1

    r1

    s0

    f0

    f

    7 0 0 0 0

    1 5 0 0

    2 10 0 0 18 0 0 0 0

    1 5 0 0

    2 10 0 0 1

    9 0 0 0 0

    1 5 0 0

    2 10 0 0 1

    10 0 0 0 0

    1 5 0 0

    2 10 0 0 1

  • 8/10/2019 Rsh Qam11 Module02

    14/24

    M2-14 MODULE 2 DYNAMIC PROGRAMMING

    The solution for stage 1 shows the best decision, the return for this stage, and the total re-

    turn, given all possible number of tons available (0 to 10 tons). Using the results of stage 1, we

    can now proceed to stage 2. The solution for stage 2 is as follows:

    STAGE 2

    S2

    d2

    r2

    s1

    f1

    f2

    0 0 0 0 0 0

    1 0 0 1 0 0

    2 0 0 2 2 5

    3 0 0 3 5 5

    1 8 0 0 8

    4 0 0 4 10 10

    1 8 1 0 8

    5 0 0 5 10 10

    1 8 2 5 13

    6 0 0 6 10 10

    1 8 3 5 13

    2 16 0 0 16

    STAGE 2

    S2

    d2

    r2

    s1

    f1

    f2

    7 0 0 7 10 10

    1 8 4 10 18

    2 16 1 0 16

    8 0 0 8 10 10

    1 8 5 10 18

    2 16 2 5 21

    9 0 0 9 10 10

    1 8 6 10 18

    2 16 3 5 21

    10 0 0 10 10 10

    1 8 7 10 18

    2 16 4 10 26

    The solution for stage 2 is found in exactly the same way as for stage 1. At stage 2 we still

    have to consider all possible number of tons available (from 0 to 10). See the column (the first

    column). At stage 2 (item 3) we still only have 2 units that can be shipped. Thus, (second col-

    umn) can range from 0 to a maximum of 2. The return for each and combination at stage 2,

    is shown in the third column. These numbers are the profit per item at this stage, 8, times the

    number of items shipped. Because items shipped at stage 2 can be 0, 1, or 2, the profit at this

    stage can be 0, 8, or 16. The return for stage 2 can also be computed from the return function:

    Now look at the fourth column, which lists the number of items available after stage 2.This is also the number of items available for stage 1. To get this number, we have to subtract

    the number of tons we are shipping at stage 2 (which is the tonnage per unit times the number of

    units) from the number of tons available before stage 2 Look at the row in which is 6 and

    is 1. We have 6 tons available before stage 2, and we are shipping 1 item, which weighs 3

    tons. Thus we will have 3 tons still available after this stage. The values can also be deter-

    mined using the transformation function, which is

    The last two columns of stage 2 contain the total return. The return afterstage 1 and before

    stage 2 is These are the same values that appeared under the column for stage 1. The re-

    turn after stage 2 is It is equal to the return from stage 2 plus the total return before stage 2.

    Stage 3 is solved in the same way as stages 1 and 2. The following table presents the

    solution for stage 3; look at each row and make sure that you understand the meaning of each

    value.

    f2.

    f1f1.

    s1 = s2 - 3d2.

    s1

    d2

    s2(s2).

    s1,

    r2 = 8d2.

    r2,

    d2s2

    d2

    s2

  • 8/10/2019 Rsh Qam11 Module02

    15/24

  • 8/10/2019 Rsh Qam11 Module02

    16/24

    M2-16 MODULE 2 DYNAMIC PROGRAMMING

    We start with shipping 6 units at stage 4. Note that is 4 from the stage 4 calculations,

    given that is 6. We use this value of 4 and go to the stage 3 calculations. We find the rows in

    which is 4 and pick the row with the highest total return, In this row is 0 items with a

    total return of 10. As a result, the number of units available, is still 4. We next go to the

    calculations for stage 2 and then stage 1 in the same way. This gives us the optimal solution al-

    ready described. see if you can find one of the alternate optimal solutions.

    s2,(f3)

    d3f3.s3

    d4

    s3

    Key Equations

    (M2-1)

    The input to stage n. This is also the output from stage

    (M2-2)

    The decision at stage n.

    (M2-3)

    The return function, usually profit or loss, at stage n.

    (M2-4)

    The output from stage n. This is also the input to stagen - 1.

    sn-1 = Outputfromstagen

    rn = Returnatstagen

    dn = Decisionatstagen

    n + 1.

    sn = Inputtostagen (M2-5)

    A transformation function that allows us to go from

    one stage to another.

    (M2-6)

    The general relationship that shows how the outputfrom any stage is a function of the input to the stage

    and the decisions made at that stage.

    (M2-7)

    This equation gives the total return (profit or costs) at

    any stage. It is obtained by summing the return at each

    stage, rn.

    fn = Totalreturnatstagen

    sn-1 = tn1sn, dn2

    tn = Transformationfunctionatstagen

    Summary

    Dynamic programming is a flexible, powerful technique. A

    large number of problems can be solved using this technique,

    including the shortest-route and knapsack problems. The

    shortest-route problem finds the path through a network that

    minimizes total distance traveled, and the knapsack problem

    maximizes the value or return. An example of a knapsack prob-

    lem is to determine what to ship in a cargo plane to maximize

    total profits given the weight and size constraints of the cargo

    plane. The dynamic programming technique requires four

    steps: (1) Divide the original problem into stages, (2) solve the

    last stage first, (3) work backward solving each subsequent

    stage, and (4) obtain the optimal solution after all stages havebeen solved.

    Dynamic programming requires that we specify stages,

    state variables, decision variables, decision criteria, an optimal

    policy, and a transformation function for each specific prob-

    lem. Stages are logical subproblems. State variables are possi-

    ble input values or beginning conditions. The decision

    variables are the alternatives that we have at each stage, such

    as which route to take in a shortest-route problem. The deci-

    sion criterion is the objective of the problem, such as finding

    the shortest route or maximizing total return in a knapsack

    problem. The optimal policy helps us obtain an optimal solu-

    tion at any stage, and the transformation function allows us to

    go from one stage to the next.

    Glossary

    Decision Criterion A statement concerning the objective of

    a dynamic programming problem.

    Decision Variable The alternatives or possible decisions that

    exist at each stage of a dynamic programming problem.

    Dynamic Programming A quantitative technique that

    works backward from the end of the problem to the begin-

    ning of the problem in determining the best decision for a

    number of interrelated decisions.

    Optimal Policy A set of decision rules, developed as a result

    of the decision criteria, that gives optimal decisions at any

    stage of a dynamic programming problem.

    Stage A logical subproblem in a dynamic programming

    problem.

    State Variable A term used in dynamic programming to

    describe the possible beginning situations or conditions

    of a stage.Transformation An algebraic statement that shows the

    relationship between stages in a dynamic programming

    problem.

  • 8/10/2019 Rsh Qam11 Module02

    17/24

    SOLVED PROBLEM M2-

    Solved Problem

    Solved Problem M2-1

    Lindsey Cortizan would like to travel from the university to her hometown over the holiday season.

    road map from the university (node 1) to her home (node 7) is shown in Figure M2.9. What is the b

    route that will minimize total distance traveled?

    1

    2

    3

    4

    5

    6

    7

    20

    18

    22

    39

    28

    36

    18

    13

    10

    FIGURE M2.9

    Road Map forLindsey Cortizan

    1

    4

    3

    2

    6

    5

    7

    20

    18

    22

    39

    28

    36

    18

    13

    10

    Stage 3 Stage 2 Stage 1

    50 41

    28

    52

    10

    13

    FIGURE M2.10

    Solution for theLindsey CortizanProblem

    Solution

    The solution to this problem is identical to the one presented earlier in the chapter. First, the problem

    broken into three stages. See the network in Figure M2.10. Working backward, we start by solving sta

    3. The closest and only distance from node 6 to 7 is 13, and the closest and only distance from node 5

    node 7 is 10. We proceed to stage 2. The minimum distances are 52, 41, and 28 from node 4, node

    and node 2 to node 7. Finally, we complete stage 3. The optimal solution is 50. The shortest route

    1257 and is shown in the following network. This problem can also be solved using tables, as show

    following the network solution.

    Problem type: Minimization network

    Number of stages: 3

    Transition function type:

    Recursion function type: fn = rn + fn-1

    sn-1 = sn - dn

  • 8/10/2019 Rsh Qam11 Module02

    18/24

    M2-18 MODULE 2 DYNAMIC PROGRAMMING

    STAGE NUMBER OF DECISIONS

    3 3

    2 4

    1 2

    STAGE STARTING NODE ENDING NODE RETURN VALUE

    3 1 2 22

    1 3 18

    1 4 20

    2 2 5 18

    2 6 36

    3 6 28

    4 6 39

    1 5 7 10

    6 7 13

    STAGE 1

    s1

    d1

    r1

    s0

    f0

    f1

    6 67 13 7 0 13

    5 57 10 7 0 10

    STAGE 2

    s2

    d2

    r2

    s1

    f1

    f2

    4 46 39 6 13 52

    3 36 28 6 13 41

    2 26 36 6 13 49

    25 18 5 10 28

    STAGE 3

    s3

    d3

    r3

    s2

    f2

    f3

    1 14 20 4 52 72

    13 18 3 41 59

    12 22 2 28 50

    FINAL SOLUTION

    STAGE OPTIMAL DECISION OPTIMAL RETURN

    3 12 22

    2 25 18

    1 57 10

    Total 50

  • 8/10/2019 Rsh Qam11 Module02

    19/24

    SELF-TEST M2-

    Self-Test

    Before taking the self-test, refer back to the learning objectives at the beginning of the module, the notes in the margins, and

    the glossary at the end of the module. Use the key at the back of the book to correct your answers. Restudy pages that correspond to any questions that you answered incorrectly or material you feel uncertain about.

    c. a decision variable.

    d. the optimal policy.e. a stage.

    9. The statement that the distance from the beginning sta

    is equal to the distance from the preceding stage to t

    last node plus the distance from the given stage to the p

    ceding stage is called

    a. the transformation.

    b. state variables.

    c. decision variables.

    d. the optimal policy.

    e. stages.

    10. In dynamic programming, is

    a. the input to the stage n.

    b. the decision at stage n.

    c. the return at stage n.

    d. the output of stage n.

    e. none of the above.

    11. The distance from the beginning stage is equal to the d

    tance from the preceding stage to the last node plus t

    distance for the given stage to the preceding stage. Th

    relationship is used to solve which type of problem?

    a. knapsack

    b. JIT

    c. shortest route

    d. minimal spanning tree

    e. maximal flow12. In using dynamic programming to solve a shortest-ro

    problem, the distance from one point to the next would

    called a

    a. state.

    b. stage.

    c. return.

    d. decision.

    13. In using dynamic programming to solve a shortest-ro

    problem, the decision variables at one stage of the pro

    lem would be

    a. the distances from one node to the next.

    b. the possible arcs or routes that can be selected.c. the number of possible destination nodes.

    d. the entering nodes.

    14. In using dynamic programming to solve a shortest-ro

    problem, the entering nodes would be called

    a. the stages.

    b. the state variables.

    c. the returns.

    d. the decision variables.

    sn

    1. Dynamic programming divides problems into

    a. nodes.b. arcs.

    c. decision stages.

    d. branches.

    e. variables.

    2. Possible beginning situations or conditions of a dynamic

    programming problem are called

    a. stages.

    b. state variables.

    c. decision variables.

    d. optimal policy.

    e. transformation.

    3. The statement concerning the objective of a dynamic pro-

    gramming problem is called

    a. stages.

    b. state variables.

    c. decision variables.

    d. optimal policy.

    e. decision criterion.

    4. The first step of a dynamic programming problem is to

    a. define the nodes.

    b. define the arcs.

    c. divide the original problem into stages.

    d. determine the optimal policy.

    e. none of the above.

    5. In working a problem with dynamic programming, weusually

    a. start at the first part of the problem and work forward

    to the next parts.

    b. start at the end of the problem and work backward.

    c. start at the most expensive part of the problem.

    d. start at the least expensive part of the problem.

    6. An algebraic statement that reveals the relationship be-

    tween stages is called

    a. the transformation.

    b. state variables.

    c. decision variables.

    d. the optimal policy.e. the decision criterion.

    7. In this chapter, dynamic programming is used to solve

    what type of problem?

    a. quantity discount

    b. just-in-time inventory

    c. shortest route

    d. minimal spanning tree

    e. maximal flow

    8. In dynamic programming terminology, a period or logical

    subproblem is called

    a. the transformation.

    b. a state variable.

  • 8/10/2019 Rsh Qam11 Module02

    20/24

    M2-20 MODULE 2 DYNAMIC PROGRAMMING

    Discussion Questions and Problems

    Discussion Questions

    M2-1 What is a stage in dynamic programming?

    M2-2 What is the difference between a state variable and a

    decision variable?

    M2-3 Describe the meaning and use of a decision crite-

    rion.M2-4 Do all dynamic programming problems require an

    optimal policy?

    M2-5 Why is transformation important for dynamic pro-

    gramming problems?

    Problems

    M2-6 Refer to Figure M2.1. What is the shortest route be-

    tween Rice and Dixieville if the road between Hope

    and Georgetown is improved and the distance is re-

    duce to 4 miles?

    M2-7 Due to road construction between Georgetown and

    Dixieville, a detour must be taken through countryroads (Figure M2.1). Unfortunately, this detour has

    increased the distance from Georgetown to Dix-

    ieville to 14 miles. What should George do? Should

    he take a different route?

    M2-8 The Rice brothers have a gold mine between Rice

    and Brown. In their zeal to find gold, they have

    blown up the road between Rice and Brown. The

    road will not be in service for five months. What

    should George do? Refer to Figure M2.1.

    M2-9 The Rice brothers (Figure M2.1) wish to determine

    the potential savings from using the shortest route

    from Rice to Dixieville rather than just randomly se-lecting any route. Use dynamic programming to find

    the longest route from Rice to Dixieville. How much

    farther is this than the shortest route?

    M2-10 Solve the shortest-route problem for the network

    shown in Figure M2.11.

    M2-11 Solve the shortest-route problem for the network

    shown in Figure M2.12.

    M2-12 Mail Express, an overnight mail service, delivers

    mail to customers throughout the United States,Canada, and Mexico. Fortunately, Mail Express has

    additional capacity on one of its cargo planes. To

    maximize profits, Mail Express takes shipments

    from local manufacturing plants to warehouses for

    other companies. Currently, there is room for an-

    other 6 tons. The following table shows the items

    that can be shipped, their weights, the expected

    profit for each, and the number of available parts.

    How many units of each item do you suggest that

    Mail Express ship?

    1 5

    2

    3

    7

    4

    8

    6

    9

    5

    4

    11

    10

    4

    12

    6

    10

    4

    6

    2

    4

    6

    FIGURE M2.11

    (for Problem M2-10)

    ITEMS TO BE SHIPPED

    WEIGHT NUMBERITEM (TONS) PROFIT/UNIT AVAILABLE

    1 1 $3 6

    2 2 9 1

    3 3 8 2

    4 1 2 2

    M2-13 Leslie Bessler must travel from her hometown to

    Denver to see her friend Austin. Given the road map

    of Figure M2.13, what route will minimize the dis-

    tance that Leslie travels?

    M2-14 An air cargo company has the shipping requirementsshown in the accompanying table. Two planes are

    available with a total capacity of 11 tons. How many

    of each item should be shipped to maximize profits?

  • 8/10/2019 Rsh Qam11 Module02

    21/24

    DISCUSSION QUESTIONS AND PROBLEMS M2-

    1

    3

    2

    6

    5

    4

    7

    8

    9

    10

    11

    3

    4

    5

    3

    2

    2

    7

    5

    2

    2

    4 3

    3

    3

    6

    FIGURE M2.12

    (for Problem M2-11)

    M2-15 Because of a new manufacturing and packaging pro-

    cedure, the weight of item 2 in Problem M2-14 can

    be cut in half. Does this change the number or types

    of items that can be shipped by the air transport

    company?

    M2-16 What is the shortest route through the network in

    Figure M2.14?

    M2-17 Refer to Problem M2-16. The road between node 6

    and node 11 is no longer in service due to construc-

    tion. What is the shortest route, given this situation?

    M2-18 In Problem M2-16, what is the shortest distan

    from node 6 to the ending node? How does th

    change if the road between node 6 and node 11 is

    longer in service (see Problem M2-17)?

    M2-19 Paige Schwartz is planning to go camping in BBend National Park. She will carry a backpack w

    food and other items, and she wants to carry no mo

    than 18 pounds of food. The four possible food item

    she is considering each have certain nutritional ing

    dients. These items and their weights are as follow

    How many of each item should paige carry if s

    wishes to maximize the total nutritional units?

    1

    4

    3

    2

    5

    6

    9

    8

    7

    10

    11

    12

    2

    1

    3 3

    4

    4

    3 3

    2

    3

    2

    3

    4

    2

    3

    3

    FIGURE M2.13

    (for Problem M2-13)

    ITEMS TO BE SHIPPED

    WEIGHT NUMBERITEM (TONS) PROFIT/UNIT AVAILABLE

    1 1 $3 62 2 9 1

    3 3 8 2

    4 2 5 5

    5 5 8 6

    6 1 2 2

    ITEM WEIGHT (POUNDS) NUTRITIONAL UNIT

    A 3 600

    B 4 1,000

    C 5 1,500

    D 2 300

  • 8/10/2019 Rsh Qam11 Module02

    22/24

    M2-22 MODULE 2 DYNAMIC PROGRAMMING

    1 3

    2

    47

    8

    6

    59

    10

    11

    12

    13

    16

    15

    14

    17

    18

    19 20

    4

    2

    3 3

    3

    4

    3

    4

    3

    3

    4

    3

    3

    2

    4

    3

    4

    4

    4

    3

    2

    5

    2

    2

    2

    FIGURE M2.14

    (for Problem M2-16)

    Case Study

    United Trucking

    Like many other trucking operations, United Trucking gotstarted with one truck and one ownerJudson Maclay. Judson

    is an individualist and has always liked to do things his way.

    He was a fast driver, and many people called the 800 number

    on the back of his truck when he worked for Hartmann Truck-

    ing. After two years with Hartmann and numerous calls about

    his bad driving, Judson decided to go out on his own. He

    started United Trucking.

    In the early days of United Trucking, Judson was the only

    driver. On the back of his truck was the message: How do you

    like my driving? Call 1-800-AMI-FAST. He was convinced

    that some people actually tried to call the number. Soon, a num-

    ber of truck operators had the same or similar messages on the

    back of their trucks. After three years of operation, Judson had

    15 other trucks and drivers working for him. He traded his driv-

    ing skills for office and management skills. Although 1-800-

    AMI-FAST was no longer visible, Judson decided to never

    place an 800 number on the back of any of his trucks. If some-

    one really wanted to complain, they could look up United

    Trucking in the phone book.

    Judson liked to innovate with his trucking company. He

    knew that he could make more money by keeping his trucks

    full. Thus he decided to institute a discount trucking service. He

    gave substantial discounts to customers that would accept deliv-

    ery to the West Coast within two weeks. Customers got a great

    price, and he made more money and kept his trucks full. Overtime, Judson developed steady customers that would usually

    have loads to go at the discounted price. On one shipment, hehad an available capacity of 10 tons in several trucks going to

    the West Coast. Ten items can be shipped at discount. The

    weight, profit, and number of items available are shown in the

    following table:

    WEIGHTITEM (TONS) PROFIT/UNIT AVAILABLE

    1 1 $10 2

    2 1 10 1

    3 2 5 3

    4 1 7 20

    5 3 25 2

    6 1 11 1

    7 4 30 2

    8 3 50 1

    9 1 10 2

    10 1 5 4

    Discussion Questions

    1. What do you recommend that Judson do?

    2. If the total available capacity were 20 tons, how wouldthis change Judsons decision?

    See our Internet home page at www.pearsonhighered.com/render for this additional case study:

    Briarcliff Electronics.

    Internet Case Study

  • 8/10/2019 Rsh Qam11 Module02

    23/24

    BIBLIOGRAPHY M2-

    Bibliography

    Bellman, R. E.Dynamic Programming. Princeton, NJ: Princeton University

    Press, 1957.

    Bourland, Karla. Parallel-Machine Scheduling with Fractional Operator Re-

    quirements,IEE Transactions (September 1994): 56.

    Elmaghraby, Salah. Resource Allocation via Dynamic Programming,

    European Journal of Operations Research (January 1993): 199215.

    El-Rayes, Khaled, et al. Optimized Scheduling for Highway Construction,Transactions of AACE International (February 1997): 311315.

    Hillard, Michael R., et al. Scheduling the Operation Desert Storm Airlift: An

    Advanced Automated Scheduling Support System,Interfaces 21, 1

    (JanuaryFebruary 1992): 131146.

    Howard, R. A.Dynamic Programming. Cambridge, MA: The MIT Press,

    1960.

    Ibarake, Toshihide, and Yuichi Nakamura. A Dynamic Programming Meth

    for Single Machine Scheduling,European Journal of Operations

    Research (July 1994): 7282.

    Idem, Fidelis, and A. M. Reisman. An Approach to Planning for Physician

    Requirements in Developing Countries Using Dynamic ProgrammingOperations Research (July 1990): 607618.

    Stokes, Jeffrey, et al. Optimal Marketing of Nursery Crops from Container

    Based Production Systems,American Journal of Agricultural

    Economics (February 1997): 235246.

  • 8/10/2019 Rsh Qam11 Module02

    24/24