mod02

7/29/2019 mod02

http://slidepdf.com/reader/full/mod02 1/23

DYNAMIC PROGRAMMINGDYNAMIC PROGRAMMING

L E A R N I N G O B J E C T I V E S

After completing this module, students will be able to:

1. Understand the overall approach of dynamic program-ming.

2. Use dynamic programming to solve the shortest-routeproblem.

3. Develop dynamic programming stages.

4. Describe important dynamic programming terminol-ogy.

5. Describe the use of dynamic programming in solvingknapsack problems.

.

Summary • Glossary • Key Equations • Solved Problem • Self-Test •Discussion Questions and Problems • Case Study: United Trucking •Internet Case Study • Bibliography

M O D U L E O U T L I N EM2.1 Introduction

M2.2 Shortest-Route Problem Solved by DynamicProgramming

M2.3 Dynamic Programming Terminology

M2.4 Dynamic Programming Notation

M2.5 Knapsack Problem

MODU L E 2

7/29/2019 mod02


M2-2 MODULE 2 Dynamic Programming

Dynamic programming

breaks a difficult problem into

subproblems.

The first step is to divide the

problem into subproblems or

stages.

The second step is to solve the

last stage—stage 1.

M2.1 INTRODUCTION

Dynamic programming is a quantitative analysis technique that has been applied to large,

complex problems that have sequences of decisions to be made. Dynamic programmingdivides problems into a number of decision stages ; the outcome of a decision at one stage

affects the decision at each of the next stages. The technique is useful in a large numberof multiperiod business problems, such as smoothing production employment, allocat-

ing capital funds, allocating salespeople to marketing areas, and evaluating investmentopportunities.Dynamic programming differs from linear programming in two ways. First, there is no

algorithm (like the simplex method) that can be programmed to solve all problems.Dynamic programming is, instead, a technique that allows us to break up difficult prob-

lems into a sequence of easier subproblems, which are then evaluated by stages.Second, lin-ear programming is a method that gives single-stage (one time period) solutions. Dynamic

programming has the power to determine the optimal solution over a one-year time hori-zon by breaking the problem into 12 smaller one-month horizon problems and to solveeach of these optimally. Hence, it uses a multistage approach.

Solving problems with dynamic programming involves four steps:

Four Steps of Dynamic Programming

1. Divide the original problem into subproblems called stages.

2. Solve the last stage of the problem for all possible conditions or states.

3. Working backward from the last stage, solve each intermediate stage. This is done by determining optimal policies from that stage to the end of the problem (last stage).

4. Obtain the optimal solution for the original problem by solving all stages sequentially.

In this module we show how to solve two types of dynamic programming problems:network and nonnetwork. The shortest-route problem is a network problem that can be

solved by dynamic programming. The knapsack problem is an example of a nonnetwork problem that can be solved using dynamic programming.

M2.2SHORTEST-ROUTE PROBLEM SOLVED BY DYNAMIC PROGRAMMING

George Yates is about to make a trip from Rice, Georgia (1), to Dixieville, Georgia (7).George would like to find the shortest route. Unfortunately, there are a number of small

towns between Rice and Dixieville. His road map is shown in Figure M2.1.The circles on the map, called nodes , represent cities such as Rice, Dixieville, Brown,

and so on. The arrows, called arcs , represent highways between the cities. The distance inmiles is indicated along each arc. This problem can, of course, be solved by inspection. Butseeing how dynamic programming can be used on this simple problem will teach you how

to solve larger and more complex problems.

Step 1: The first step is to divide the problem into subproblems or stages. Figure M2.2

reveals the stages of this problem. In dynamic programming, we usually start with the lastpart of the problem, stage 1, and work backward to the beginning of the problem or net-

work, which is stage 3 in this problem. Table M2.1 summarizes the arcs and arc distancesfor each stage.

Step 2: We next solve stage 1, the last part of the network. Usually, this is trivial. We find theshortest path to the end of the network, node 7 in this problem. At stage 1, the shortest paths,

7/29/2019 mod02


M2.2: Shortest-Route Problem Solved by Dynamic Programming M2-3

10 Miles

10 Miles

1 2 M i l e

s

4 M i l e sBrown

2 M i l e

s

1 4 M i l e s

5 Miles

4 M i l e

s

2 M i l e s

Lake City

Hope

Athens

Georgetown

Dixieville

4

1 3

2

5

6

7

6 M i l e s

Rice

F I G U R E M 2 . 1 Highway Map between Rice and Dixieville

1 3

2

4

6

5

7

Stage 3

A Branch

Stage 2 Stage 1

A Node

45

2

10

12

6

4 10

2

14

F I G U R E M 2 . 2

Three Stages for the

George Yates Problem

STAGE ARC ARC DISTANCE

1 5–7 14

6–7 2

2 4–5 10

3–5 12

3–6 6

2–5 4

2–6 10

3 1–4 4

1–3 5

1–2 2

T A B L E M 2 . 1

Distance Along Each Arc

7/29/2019 mod02



45

2

10

12

6

4 10

2

14

Minimum Distance to Node 7 from Node 6


1 3

2

4

6

5

7

2

14

F I G U R E M 2 . 3

Solution for the One-Stage

Problem

Step 3 involves moving back-

ward solving intermediate

stages.

45

2

10Minimum Distance to Node 7 from Node 3

12

6

4 10

2

14



1 3

2

4

6

5

7

8

12

14

2

24

F I G U R E M 2 . 4

Solution for the Two-Stage

Problem

from node 5 to node 6, to node 7 are the only paths. Also note in Figure M2.3 that the mini-

mum distances are enclosed in boxes by the entering nodes to stage 1, node 5 and node 6. Theobjective is to find the shortest distance to node 7. The following table summarizes this proce-

dure for stage 1.As mentioned previously, the shortest distance is the only distance at stage 1.

STAGE 1

BEGINNING SHORTEST DISTANCE ARCS ALONGNODE TO NODE 7 THIS PATH

5 14 5–7

6 2 6–7

Step 3: Moving backward, we now solve for stages 2 and 3. At stage 2 we will use Figure

M2.4.

7/29/2019 mod02


M2.2: Shortest-Route Problem Solved by Dynamic Programming M2-5

1 3

2

4

6

5

7

45

2

10

8

12

6

4 10

2

14

24


12

14

2

13

F I G U R E M 2 . 5

Solution for the Three-

Stage Problem

If we are at node 4, the shortest and only route to node 7 is arcs 4–5 and 5–7. At node

3, the shortest route is arcs 3–6 and 6–7 with a total minimum distance of 8 miles. If we areat node 2, the shortest route is arcs 2–6 and 6–7 with a minimum total distance of 12 miles.This information is summarized in the stage 2 table:

STAGE 2

BEGINNING SHORTEST DISTANCE ARCS ALONG

NODE TO NODE 7 THIS PATH

4 24 4–5

5–7

3 8 3–6

6–7

2 12 2–6

6–7

The solution to stage 3 can be completed using the accompanying table and the net-work in Figure M2.5.

STAGE 3

BEGINNING SHORTEST DISTANCE ARCS ALONGNODE TO NODE 7 THIS PATH

1 13 1–3

3–6

6–7

7/29/2019 mod02



The fourth and final step is to

find the optimal solution after

all stages have been solved.

Managing a nursery that produces ornamental plants is diffi-

cult. In most cases, ornamental plants increase in value with

increased growth. This value-added growth makes it difficult to

determine when to harvest the plants and place them on the

market. When plants are marketed earlier, revenues are gener-

ated earlier and the costs associated with plant growth are min-

imized. On the other hand, delaying the harvesting of the orna-

mental plants usually results in higher prices. But are the

additional months of growth and costs worth the delay?

In this case, dynamic programming was used to determine

the optimal growth stages for ornamental plants. Each stage was

associated with a possible growth level. The state variables

included acres of production of ornamental plants and carry-

over plants from previous growing seasons. The objective of the

dynamic programming problem was to maximize the after-tax

cash flow. The taxes included self-employment, federal income,

earned income credit, and state income taxes.

The solution was to produce one- and three-gallon con-

tainers of ornamental plants. The one-gallon containers are

sold in the fall and carried over for spring sales.Any one-gallon

containers not sold in the spring are combined into three-gal-

lon container products for sale during the next season. Using

dynamic programming helps to determine when to harvest to

increase after-tax cash flow.

Source: Stokes, Jeffrey et al., “Optimal Marketing of Nursery Crops fromContainer-Based Production Systems,” American Journal of Agricultural Economics (February 1997): 235–246.

IN ACTION Dynamic Programming in Nursery Production Decisions

Step 4: To obtain the optimal solution at any stage, all we consider are the arcs to the next

stage and the optimal solution at the next stage. For stage 3, we only have to consider thethree arcs to stage 2 (1–2, 1–3, and 1–4) and the optimal policies at stage 2, given in a pre-vious table. This is how we arrived at the preceding solution.When the procedure is under-

stood, we can perform all the calculations on one network. You may want to study the rela-tionship between the networks and the tables because more complex problems are usually

solved by using tables only.

M2.3 DYNAMIC PROGRAMMING TERMINOLOGY

Regardless of the type or size of a dynamic programming problem, there are some important

terms and concepts that are inherent in every problem. Some of the more important follow:

1. Stage: a period or a logical subproblem.

2. State variables: possible beginning situations or conditions of a stage. These have alsobeen called the input variables.

3. Decision variables: alternatives or possible decisions that exist at each stage.

4. Decision criterion: a statement concerning the objective of the problem.

5. Optimal policy: a set of decision rules, developed as a result of the decision criteria,that gives optimal decisions for any entering condition at any stage.

6. Transformation: normally, an algebraic statement that reveals the relationshipbetween stages.

In the shortest-route problem, the following transformation can be given:

This relationship shows how we are able to go from one stage to the next in solving forthe optimal solution to the shortest-route problem. In more complex problems, we can use

symbols to show the relationship between stages.State variables, decision variables, the decision criterion, and the optimal policy can be

determined for any stage of a dynamic programming problem. This is done here for stage 2

of the George Yates shortest-route problem.

distance from the distance from the beginning distance from the

beginning of a given of the stage given stage to the

stage to the last node to the last node previous stage

= + previous

7/29/2019 mod02


M2.3: Dynamic Programming Terminology M2-7

1. State variables for stage 2 are the entering nodes, which are

(a) node 2

(b) node 3

(c) node 4

2. Decision variables for stage 2 are the following arcs or routes:

(a) 4–5

(b) 3–5

(c) 3–6

(d) 2–5

(e) 2–6

3. The decision criterion is the minimization of the total distances traveled.

4. The optimal policy for any beginning condition is shown in Figure M2.6 and the fol-

lowing table:

GIVEN THIS ENTERING THIS ARC WILL MINIMIZECONDITION TOTAL DISTANCE TO NODE 7

2 2–6

3 3–6

4 4–5

1 3

2

4

6

5

7

10

8

12

6

4 10

24

12

14

2

Statevariablesare the entering nodes.

The optimal policy is the arc,

for any entering node, that will minimize total distance to the destination at this stage.

Decisionvariablesare all the arcs.

Stage 2

F I G U R E M 2 . 6

Stage 2 from the Shortest-

Route Problem

7/29/2019 mod02



The input to one stage is the

output from another stage.

Stage 2

Decisiond 2

Outputs 1

Returnr 2

Inputs 2

F I G U R E M 2 . 7

Input, Decision,Output,

and Return for Stage 2 in

George Yates’s Problem

Figure M2.6 may also be helpful in understanding some of the terminology used in the

discussion of dynamic programming.

M2.4 DYNAMIC PROGRAMMING NOTATION

In addition to dynamic programming terminology, we can also use mathematical notationto describe any dynamic programming problem. This helps us to set up and solve the prob-

lem. Consider stage 2 in the George Yates dynamic programming problem first discussed inSection M2.2. This stage can be represented by the diagram shown in Figure M2.7 (as couldany given stage of a given dynamic programming problem).

As you can see, for every stage, we have an input, decision, output and return . Look again at stage 2 for the George Yates problem in Figure M2.6. The input to this stage is s 2,

which consists of nodes 2, 3, and 4. The decision at stage 2, or choosing which arc will leadto stage 1, is represented by d 2. The possible arcs or decisions are 4–5, 3–5, 3–6, 2–5 and

2–6. The output to stage 2 becomes the input to stage 1. The output from stage 2 is s 1. Thepossible outputs from stage 2 are the exiting nodes, nodes 5 and 6. Finally, each stage has areturn. For stage 2, the return is represented by r 2. In our shortest-route problem, the

return is the distance along the arcs in stage 2. These distances are 10 miles for arc 4–5, 12miles for arc 3–5, 6 miles for arc 3–6, 4 miles for arc 2–5 and 10 miles for arc 2–6. The same

notation applies for the other stages and can be used at any stage. In general, we use the fol-

lowing notation for these important concepts:(M2-1)

(M2-2)

(M2-3)

You should also note that the input to one stage is also the output from another stage.For example, the input to stage 2, s 2, is also the output from stage 3 (see Figure M2.7). This

leads us to the following equation:

(M2-4)s n n − =1 output from stage

r n n = return at stage

d n n = decision at stage

s n n = input to stage

An input, decision, output,

and return are specified for

each stage.

7/29/2019 mod02


M2.5: Knapsack Problem M2-9

A transformation function

allows us to go from one stage

to another.

The total return function

allows us to keep track of profits and costs.

The final concept is transformation . The transformation function allows us to go from

one stage to another.The transformation function for stage 2, t 2, converts the input to stage2, s 2, and the decision made at stage 2, d 2, to the output from stage 2, s 1. Because the trans-formation function depends on the input and decision at any stage, it can be represented as

t 2 (s 2, d 2). In general, the transformation function can be represented as follows:

(M2-5)

The following general formula allows us to go from one stage to another using the trans-formation function:

(M2-6)

Although this equation may seem complex, it is really just a mathematical statement of thefact that the output from a stage is a function of the input to the stage and any decisions

made at that stage. In the George Yates shortest-route problem, the transformation func-tion consists of a number of tables. These tables show how we could progress from onestage to another in order to solve the problem. For more complex problems, we need to use

dynamic programming notation instead of tables.Another useful quantity is the total return at any stage. The total return allows us to

keep track of the total profit or costs at each stage as we solve the dynamic programming

problem. It can be given as follows:(M2-7)

M2.5 KNAPSACK PROBLEM

The “knapsack problem” involves the maximization or the minimization of a value, such asprofits or costs. Like a linear programming problem, there are restrictions. Imagine a knap-

sack or pouch that can only hold a certain weight or volume. We can place different typesof items in the knapsack. Our objective is to place items in the knapsack to maximize totalvalue without breaking the knapsack because of too much weight or a similar restriction.

Types of Knapsack Problems

There are many kinds of problems that can be classified as knapsack problems. Choosingitems to place in the cargo compartment of an airplane and selecting which payloads to put

on the next NASA Space Shuttle are examples. The restriction can be volume, weight, orboth. Some scheduling problems are also knapsack problems.For example,we may want to

determine which jobs to complete in the next two weeks. The two-week period is the knap-sack, and we want to load it with jobs in such a way as to maximize profits or minimizecosts. The restriction is the number of days or hours during the two-week period.

Roller’s Air Transport Service ProblemRob Roller owns and operates Roller’s Air Transport Service, which ships cargo by plane tomost large cities in the United States and Canada. The remaining capacity for one of the

flights from Seattle to Vancouver is 10 tons. There are four different items that Rob can shipbetween Seattle and Vancouver. Each item has a weight in tons, a net profit in thousands of

dollars, and a total number of that item that is available for shipping. This information ispresented in Table M2.2.

Problem Setup Roller’s Air Transport problem is an ideal problem to solve using dynamicprogramming. Stage 4 will be item 1, stage 3 will be item 2, stage 2 will be item 3, and stage

f n n = total return at stage

s t s d n n n n −=1 ( , )

t n n = transformation function at stage

The objective of this problem

is to maximize profits.

7/29/2019 mod02



ITEM WEIGHT PROFIT/UNIT NUMBER AVAILABLE

1 1 $3 6

2 4 9 1

3 3 8 2

4 2 5 2

T A B L E M 2 . 2

Items to Be Shipped

Reducing Electric Production Costs Using Dynamic Programming➠ MODELING IN THE REAL WORLD

Defining

the Problem

Developing

a Model

Acquiring

Input Data

Developing

a Solution

Testing the

Solution

Analyzing

the Results

Implementing

the Results

The Southern Company, with service areas in Georgia, Alabama, Mississippi, and Florida, is a major

provider of electric service, with about 240 generating units. In recent years, fuel costs have increased

faster than other costs. Annual fuel costs are about $2.5 billion, representing about one-third of total

expenses for the Southern Company. The problem for the Southern Company is to reduce total fuel costs.

To deal with this fuel cost problem, the company developed a state-of-the-art dynamic programming

model. The dynamic programming model is embedded in the Wescouger optimization program, which isa computer program used to control electric generating units and reduce fuel costs through better utiliza-

tion of existing equipment.

Data were collected on past and projected electric usage. In addition, daily load/generation data were ana-

lyzed. Load/generation charts were used to investigate the fuel requirements for coal, nuclear, hydroelec-

tric, and gas/oil.

The solution of the dynamic programming model provides both short-term planning guidelines and

long-term fuel usage for the various generating units. Optimal maintenance schedules for generating

units are obtained using Wescouger.

To test the accuracy of the Wescouger optimization program, Southern used a real-time economic dis-

patch program. The results were a very close match. In addition, the company put the solution through an

acid test, in which seasoned operators compared the results against their intuitive judgment. Again, the

results were consistent.

The results were analyzed in terms of their impact on the use of various fuels, the usage of various gener-

ating units, and maintenance schedules for generating units. Analyzing the results also revealed other

needs. This resulted in a full-color screen editing routine, auxiliary programs to automate data input, and

software to generate postanalysis reports.

The Southern Company implemented the dynamic programming solution. Over a seven-year period, theresults saved the company over $140 million.

Source: S. Erwin et al. “Using an Optimization Software to Lower Overall Electric Production Costs for Southern Company,” Interfaces 21, 1 (January–February 1991): 27–41.

7/29/2019 mod02



ITEM STAGE

1 4

2 3

3 2

4 1

T A B L E M 2 . 3

Relationship Between

Items and Stages

1 will be item 4. This is shown in Table M2.3. During the solution, we will be using stage

numbers.Roller’s Air Transport problem can be represented graphically (see Figure M2.8). As

you can see, each item is represented by a stage. Look at stage 4, which is item 1. The total

weight that can be used is represented by s 4. This amount is 10 because we haven’t assignedany units to be shipped at this time. The decision at this stage is d 4 (the number of units of

item 1 we will ship). If d 4 is 1, for example, we will be shipping 1 unit of item 1. Also notethat r 4 is the return or profit at stage 4 (item 1). If we ship 1 unit of item 1, the profit will be

$3 (see Table M2.2).As mentioned previously, the decision variable d n , represents the number of units of

each item (stage) that can be shipped. Looking back at the original problem,we see that the

problem is constrained by the number of units. This is summarized in the following table:

STAGE MAXIMUM VALUE OF DECISION

4 6

3 1

2 2

1 2

The Transformation Functions Next, we need to look at the transformation function.

The general transformation function for knapsack problems follows:

Note that a n , b n , and c n are coefficients for the problem and that d n represents the decisionat stage n . This is the number of units to ship at stage s n .

s a s b d cn n n n n n−

= × + × +1 ( ) ( )

s 1Stage 2(Item 3)

Stage 3(Item 2)

d 4

s 3

r 4

s 4 Stage 4(Item 1)

s 0Stage 1(Item 4)

d 3

r 3

d 2

r 2

d 1

r 1

Decisions

Returns

s 2

F I G U R E M 2 . 8

Roller’s Air Transport

Service Problem

7/29/2019 mod02



The following chart shows the transformation coefficients for Rob Roller’s transport

problem:

COEFFICIENTS OFTRANSITION FUNCTION

STAGE a n b n c n

4 1Ϫ

1 03 1 Ϫ4 0

2 1 Ϫ3 0

1 1 Ϫ2 0

First note that s 4 is 10, the total weight that can be shipped. Because s 4 represents the firstitem, all 10 tons can be utilized. The transformation equations for the four stages are as

follows:

(a)

(b)

(c)

(d)

Consider stage 3. Equation b reveals that the number of tons still available after stage 3, s 2,

is equal to the number of tons available before stage 3, s 3, minus the number of tonsshipped at stage 3, 4d 3. In this equation, 4d 3 means that each item at stage 3 weighs 4 tons.

The Return Functions Next, we will look at the return function for each stage. This is thegeneral form for the return function:

Note that a n , b n , and c n are the coefficients for the return function. Using this general formof the return function, we can put the return function values in the following table:

COEFFICIENTS OFDECISIONS RETURN FUNCTION

STAGE LOWER UPPER a n b n c n

4 0 ≤ d n ≤ 6 0 3 0

3 0 ≤ d n ≤ 1 0 9 0

2 0 ≤ d n ≤ 2 0 8 0

1 0 ≤ d n ≤ 2 0 5 0

r a s b d cn n n n n n= × + × +( ) ( )

s s d 0 1 12= − stage 1

s s d 1 2 23= − stage 2

s s d 2 3 34= − stage 3

s s d 3 4 41= − stage 4

7/29/2019 mod02



The lower value for each decision is zero, and the upper value is the total number avail-

able. The b n coefficient is the profit per item shipped. The actual return functions are:

Stage-by-Stage Solution As you would expect, the return at any stage r n , is equal to theprofit per unit at that stage multiplied by the number of units shipped at that stage, d n .

With this information, we can solve Roller’s Air Transport problem, starting with stage 1(item 4). The following table shows the solution for the first stage. You may wish to refer toFigure M2.8 for this discussion.

STAGE 1

S 1 d 1 r 1 s 0 f 0 f 1

0 0 0 0 0 0

1 0 0 0 0 0

2 0 0 0 0 0

1 5 0 0 5

3 0 0 0 0 0

1 5 0 0 5

4 0 0 0 0 0

1 5 0 0 5

2 10 0 0 10

5 0 0 0 0 0

1 5 0 0 5

2 10 0 0 10

6 0 0 0 0 0

1 5 0 0 5

2 10 0 0 10

r d

r d

r d

r d

4 4

3 3

2 2

1 1

3

9

8

5

=

=

=

=

STAGE 1

S 1 d 1 r 1 s 0 f 0 f 1

7 0 0 0 0 0

1 5 0 0 5

2 10 0 0 10

8 0 0 0 0 0

1 5 0 0 5

2 10 0 0 10

9 0 0 0 0 0

1 5 0 0 5

2 10 0 0 10

10 0 0 0 0 0

1 5 0 0 5

2 10 0 0 10

Because we don’t know how many tons will be available at stage 1, we must consider all

possibilities. Thus the number of tons available at stage 1, s 1, can vary from 1 to 10. This isseen in the first column of numbers for stage 1. The number of units that we ship at stage1, d 1, can vary from 0 to 2. We can’t go over 2 because the number available is only 2. For

any decision we compute the return at stage 1, r 1, by multiplying the number of items

shipped by 5, the profit per item. The profit at this stage will be 0, 5, or 10, depending onwhether 0, 1 or 2 items are shipped. Note that the total return at this stage, f 1, is the same as

r 1 because this is the only stage we are considering so far. Also note that the total return

before stage 1, f 0, is 0 because this is the beginning of the solution and we are shippingnothing at this point.

We consider all possibilities.

f 1 is the total return at the

first stage. The total return

before the first stage is f 0.

7/29/2019 mod02



The solution for stage 1 shows the best decision, the return for this stage, and the total

return given all possible number of tons available (0 to 10 tons). Using the results of stage1, we can now proceed to stage 2. The solution for stage 2 is as follows:

STAGE 2

S 2 d 2 r 2 s 1 f 1 f 2

0 0 0 0 0 0

1 0 0 1 0 0

2 0 0 2 2 5

3 0 0 3 5 5

1 8 0 0 8

4 0 0 4 10 10

1 8 1 0 8

5 0 0 5 10 10

1 8 2 5 13

6 0 0 6 10 10

1 8 3 5 132 16 0 0 16

STAGE 2

S 2 d 2 r 2 s 1 f 1 f 2

7 0 0 7 10 10

1 8 4 10 18

2 16 1 0 16

8 0 0 8 10 10

1 8 5 10 18

2 16 2 5 21

9 0 0 9 10 10

1 8 6 10 18

2 16 3 5 21

10 0 0 10 10 10

1 8 7 10 182 16 4 10 26

The solution for stage 2 is found in exactly the same way as for stage 1. At stage 2 we

still have to consider all possible number of tons available (from 0 to 10). See the s 2 column(the first column).At stage 2 (item 3) we still only have 2 units that can be shipped. Thus d 2(second column) can range from 0 to a maximum of 2. The return for each s 2 and d 2 com-

bination at stage 2, r 2, is shown in the third column. These numbers are the profit per itemat this stage, 8, times the number of items shipped. Because items shipped at stage 2 can be

0, 1, or 2, the profit at this stage can be 0, 8, or 16. The return for stage 2 can also be com-puted from the return function: r 2 = 8d 2.

Now look at the fourth column, s 1, which lists the number of items available after stage2. This is also the number of items available for stage 1. To get this number, we have to sub-tract the number of tons we are shipping at stage 2 (which is the tonnage per unit times the

number of units) from the number of tons available before stage 2 (s 2). Look at the row inwhich s 2 is 6 and d 2 is 1. We have 6 tons available before stage 2,and we are shipping 1 item,

which weighs 3 tons. Thus we will have 3 tons still available after this stage. The s 1 valuescan also be determined using the transformation function, which is s 1 = s 2 – 3d 2.

The last two columns of stage 2 contain the total return. The return after stage 1 and

before stage 2 is f 1. These are the same values that appeared under the f 1 column for stage 1.The return after stage 2 is f 2. It is equal to the return from stage 2 plus the total return

before stage 2.Stage 3 is solved in the same way as stages 1 and 2. The following table presents the

solution for stage 3; look at each row and make sure that you understand the meaning of

each value.

7/29/2019 mod02



STAGE 3

s 3 d 3 r 3 s 2 f 2 f 3

0 0 0 0 0 0

1 0 0 1 0 0

2 0 0 2 2 5

3 0 0 3 8 8

4 0 0 4 10 10

1 9 0 0 9

5 0 0 5 13 13

1 9 1 0 9

6 0 0 6 16 16

1 9 2 5 14

STAGE 3

s 3 d 3 r 3 s 2 f 2 f 3

7 0 0 7 18 18

1 9 3 8 17

8 0 0 8 21 21

1 9 4 10 19

9 0 0 9 21 21

1 9 5 13 22

10 0 0 10 26 26

1 9 6 16 25

Now we solve the last stage of the problem, stage 4. The following table shows the solu-

tion procedure:

STAGE 4

s 4 d 4 r 4 s 3 f 3 f 4

10 0 0 10 26 26

1 3 9 22 25

2 6 8 21 27

3 9 7 18 27

4 12 6 16 28

5 15 5 13 28

6 18 4 10 28

The first thing to note is that we only have to consider one value for s 4, because we know the number of tons available for stage 4; s 4 must be 10 because we have all 10 tons available.

There are six possible decisions at this stage, or six possible values for d 4, because the numberof available units is 6. The other columns are computed in the same way. Note that the returnafter stage 4, f 4, is the total return for the problem. We see that the highest profit is 28. We also

see that there are three possible decisions that will give this level of profit, shipping 4, 5, or 6items. Thus we have alternate optimal solutions. One possible solution is as follows:

FINAL SOLUTION

STAGE OPTIMAL DECISION OPTIMAL RETURN

4 6 18

3 0 0

2 0 0

1 2 10

Total 8 28

7/29/2019 mod02



We start with shipping 6 units at stage 4. Note that s 3 is 4 from the stage 4 calculations,

given that d 4 is 6. We use this value of 4 and go to the stage 3 calculations. We find the rowsin which s 3 is 4 and pick the row with the highest total return, f 3. In this row d 3 is 0 itemswith a total return ( f 3) of 10. As a result, the number of units available, s 2, is still 4. We next

go to the calculations for stage 2 and then stage 1 in the same way. This gives us the optimalsolution already described. See if you can find one of the alternate optimal solutions.

SUMMARYDynamic programming is a flexible, powerful technique. Alarge number of problems can be solved using this tech-

nique, including the shortest-route and knapsack prob-lems. The shortest-route problem finds the path through a

network that minimizes total distance traveled, and theknapsack problem maximizes the value or return. Anexample of a knapsack problem is to determine what to

ship in a cargo plane to maximize total profits given theweight and size constraints of the cargo plane. The

dynamic programming technique requires four steps: (1)Divide the original problem into stages, (2) solve the last

stage first, (3) work backward solving each subsequentstage, and (4) obtain the optimal solution after all stageshave been solved.

Dynamic programming requires that we specify stages, state variables, decision variables, decision criteria,

an optimal policy, and a transformation function for eachspecific problem. Stages are logical subproblems. State

variables are possible input values or beginning conditions.The decision variables are the alternatives that we have ateach stage, such as which route to take in a shortest-route

problem. The decision criterion is the objective of theproblem, such as finding the shortest route or maximizing

total return in a knapsack problem. The optimal policy helps us obtain an optimal solution at any stage, and the

transformation function allows us to go from one stage tothe next.

GLOSSARY

Decision Criterion. A statement concerning the objective of a

dynamic programming problem.

Decision Variable. The alternatives or possible decisions that

exist at each stage of a dynamic programming problem.

Dynamic Programming. A quantitative technique that works

backward from the end of the problem to the beginning of the

problem in determining the best decision for a number of

interrelated decisions.Optimal Policy. A set of decision rules, developed as a result of

the decision criteria, that gives optimal decisions at any stage of

a dynamic programming problem.

Stage. A logical subproblem in a dynamic programming prob-

lem.

State Variable. A term used in dynamic programming to

describe the possible beginning situations or conditions of a

stage.

Transformation. An algebraic statement that shows the relation-

ship between stages in a dynamic programming problem.

KEY EQUATIONS

(M2-1)

The input to stage n . This is also the output from stagen + 1.

(M2-2)

The decision at stage n .

(M2-3)

The return function, usually profit or loss, at stage n .

(M2-4)

The output from stage n. This is also the input to stage n Ϫ 1.

s n n − =1 Output from stage

r n n

= Return at stage

n n = Decision at stage

s n n = Input to stage (M2-5)

A transformation function that allows us to go from one

stage to another.

(M2-6)

The general relationship that shows how the output from

any stage is a function of the input to the stage and thedecisions made at that stage.

(M2-7)

This equation gives the total return (profit or costs) at any

stage. It is obtained by summing the return at each stage,r n .

f n n = Total return at stage

s t s d n n n n −=1 ( , )

t n n = Transformation function at stage

7/29/2019 mod02


Solved Problem M2-17

SOLVED PROBLEM

Solved Problem M2-1

Lindsey Cortizan would like to travel from the university to her hometown over the holiday season. This

road map from the university (node 1) to her home (node 7) is shown in Figure M2.9. What is the best

route that will minimize total distance traveled?

1

2

3

4

5

6

7

20

18

22

39

28

36

18

13

10

F I G U R E M 2 . 9

Road Map for Lindsey

Cortizan

Solution

The solution to this problem is identical to the one presented earlier in the chapter. First, the problem is

broken into three stages. See the network in Figure M2.10. Working backward, we start by solving stage

3. The closest and only distance from node 6 to 7 is 13, and the closest and only distance from node 5 to

node 7 is 10.We proceed to stage 2. The minimum distances are 52, 41, and 28 from node 4, node 3, and

node 2 to node 7. Finally, we complete stage 3. The optimal solution is 50. The shortest route is 1–2–5–7

and is shown in the following network. This problem can also be solved using tables, as shown following

the network solution.

Problem type minimization network

Number of stages 3

Transition function type s n –1 = s n – d n

Recursion function type f n

= r n

+ f n –1

1

4

3

2

6

5

7

20

18

22

39

28

36

18

13

10

Stage 3 Stage 2 Stage 1

50 41

28

52

10

13

F I G U R E M 2 . 1 0

Solution for the Lindsey

Cortizan Problem

7/29/2019 mod02



STAGE NUMBER OF DECISIONS

3 3

2 4

1 2

STAGE STARTING NODE ENDING NODE RETURN VALUE3 1 2 22

1 3 18

1 4 20

2 2 5 18

2 6 36

3 6 28

4 6 39

1 5 7 10

6 7 13

STAGE 1

s 1 d 1 r 1 s 0 f 0 f 1

6 6–7 13 7 0 13

5 5–7 10 7 0 10

STAGE 2

s 2 d 2 r 2 s 1 f 1 f 2

4 4–6 39 6 13 52

3 3–6 28 6 13 41

2 2–6 36 6 13 49

2–5 18 5 10 28

STAGE 3

s 3 d 3 r 3 s 2 f 2 f 3

1 1–4 20 4 52 72

1–3 18 3 41 59

1–2 22 2 28 50

FINAL SOLUTION

STAGE OPTIMAL DECISION OPTIMAL RETURN

3 1–2 222 2–5 18

1 5–7 10

Total 50

7/29/2019 mod02


Self-Test M2-19

➠ SELF-TEST

I Before taking the self-test,refer back to the learning objectives at the beginning of the module, thenotes in the margins, and the glossary at the end of the module.

I Use the key at the back of the book to correct your answers.

I Restudy pages that correspond to any questions that you answered incorrectly or material you feeluncertain about.

1. Dynamic programming divides problems intoa. nodes.b. arcs.c. decision stages.d. branches.e. variables.

2. Possible beginning situations or conditions of a dynamicprogramming problem are calleda. stages.b. state variables.c. decision variables.d. optimal policy.e. transformation.

3. The statement concerning the objective of a dynamic pro-

gramming problem is calleda. stages.b. state variables.c. decision variables.d. optimal policy.e. decision criterion.

4. The first step of a dynamic programming problem is toa. define the nodes.b. define the arcs.c. divide the original problem into stages.d. determine the optimal policy.e. none of the above.

5. In working a problem with dynamic programming, weusually a. start at the first part of the problem and work forward

to the next parts.b. start at the end of the problem and work backward.c. start at the most expensive part of the problem.d. start at the least expensive part of the problem.

6. An algebraic statement that reveals the relationshipbetween stages is calleda. the transformation.b. state variables.c. decision variables.d. the optimal policy.e. the decision criterion.

7. In this chapter, dynamic programming is used to solvewhat type of problem?a. quantity discountb. just-in-time inventory

c. shortest-routed. minimal spanning treee. maximal flow

8. In dynamic programming terminology, a period or logicalsubproblem is calleda. the transformation.b. a state variable.

c. a decision variable.d. the optimal policy.e. a stage.

9. The statement that the distance from the beginning stage isequal to the distance from the preceding stage to the lastnode plus the distance from the given stage to the preced-ing stage is calleda. the transformation.b. state variables.c. decision variables.d. the optimal policy.e. stages.

10. In dynamic programming, s n isa. the input to the stage n .

b. the decision at stage n .c. the return at stage n .d. the output of stage n .e. none of the above.

11. The relationship that the distance from the beginning stageis equal to the distance from the preceding stage to the lastnode plus the distance for the given stage to the precedingstage is used to solve which type of problem?a. knapsack b. JITc. shortest-routed. minimal spanning treee. maximal flow

12. In using dynamic programming to solve a shortest routeproblem, the distance from one point to the next would be

called aa. state.b. stage.c. return.d. decision.

13. In using dynamic programming to solve a shortest routeproblem, the decision variables at one stage of the problemwould bea. the distances from one node to the next.b. the possible arcs or routes that can be selected.c. the number of possible destination nodes.d. the entering nodes.

14. In using dynamic programming to solve a shortest routeproblem, the entering nodes would be calleda. the stages.

b. the state variables.c. the returns.d. the decision variables.

7/29/2019 mod02



DISCUSSION QUESTIONS AND PROBLEMS

1 5

2

3

7

4

8

6

9

5

4

11

10

4

12

6

10

4

6

2

4

6

F I G U R E M 2 . 1 1

(for Problem M2-10)

Discussion Questions

M2-1 What is a stage in dynamic programming?

M2-2 What is the difference between a state variable and adecision variable?

M2-3 Describe the meaning and use of a decision criterion.

M2-4 Do all dynamic programming problems require anoptimal policy?

M2-5 Why is transformation important for dynamic pro-gramming problems?

Problems

M2-6 Refer to Figure M2.1. What is the shortest routebetween Rice and Dixieville if the road between Hopeand Georgetown is improved and the distance isreduce to 4 miles?

M2-7 Due to road construction between Georgetown andDixieville, a detour must be taken through country roads (Figure M2.1). Unfortunately, this detour hasincreased the distance from Georgetown to Dixievilleto 14 miles. What should George do? Should he take a

different route?M2-8 The Rice Brothers have a gold mine between Rice and

Brown. In their zeal to find gold, they have blown upthe road between Rice and Brown. The road will notbe in service for five months.What should George do?Refer to Figure M2.1.

M2-9 The Rice Brothers (Figure M2.1) wish to determinethe potential savings from using the shortest routefrom Rice to Dixieville rather than just randomly selecting any route. Use dynamic programming tofind the longest route from Rice to Dixieville. How much farther is this than the shortest route?

M2-10 Solve the shortest-route problem of Figure M2.11.

M2-11 Solve the shortest-route problem of Figure M2.12.

M2-12 Mail Express, an overnight mail service, delivers mailto customers throughout the United States, Canada,and Mexico. Fortunately, Mail Express has additionalcapacity on one of its cargo planes. To maximize prof-

its, Mail Express takes shipments from local manufac-turing plants to warehouses for other companies.Currently, there is room for another 6 tons. The fol-lowing table shows the items that can be shipped,their weights, the expected profit for each, and thenumber of available parts. How many units of eachitem do you suggest that Mail Express ship?

ITEMS TOBE SHIPPED

WEIGHT NUMBER

ITEM (TONS) PROFIT/UNIT AVAILABLE

1 1 $3 6

2 2 9 1

3 3 8 2

4 1 2 2

M2-13 Leslie Bessler must travel from her hometown toDenver to see her friend Austin. Given the road mapof Figure M2.13, what route will minimize the dis-tance that she travels?

M2-14 An air cargo company has the following shippingrequirements. Two planes are available with a totalcapacity of 11 tons. How many of each item should beshipped to maximize profits?

7/29/2019 mod02


Discussion Questions and Problems M2-21

1

3

2

6

5

4

7

8

9

10

11

3

4

5

3

2

2

7

5

2

2

4 3

3

3

6

F I G U R E M 2 . 1 2

(for Problem M2-11)

ITEMS TOBE SHIPPED

WEIGHT NUMBER

ITEM (TONS) PROFIT/UNIT AVAILABLE

1 1 $3 6

2 2 9 1

3 3 8 2

4 2 5 5

5 5 8 6

6 1 2 2

M2-15 Because of a new manufacturing and packaging proce-dure, the weight of item 2 in Problem M2-14 can be cutin half. Does this change the number or types of itemsthat can be shipped by the air transport company?

M2-16 What is the shortest route through the network of

Figure M2.14?M2-17 The road between node 6 and node 11 is no longer in

service due to construction. (Refer to Problem M2-16).What is the shortest route given this situation?

M2-18 In Problem M2-16, what is the shortest distance fromnode 6 to the ending node? How does this change if the road between node 6 and node 11 is no longer inservice (see Problem M2-17)?

M2-19 Paige Schwartz is planning to go camping in Big BendNational Park. She will carry a backpack with foodand other items, and she wants to carry no more than18 pounds of food. The four possible food items she isconsidering each have certain nutritional ingredients.These items and their weights are as follows:

ITEM WEIGHT (POUNDS) NUTRITIONAL UNITS

A 3 600

B 4 1,000

C 5 1,500

D 2 300

How many of each item should Paige carry if shewishes to maximize the total nutritional units?

1

4

3

2

5

6

9

8

7

10

11

12

2

1

3 3

4

4

3 3

2

3

2

3

4

2

3

3

F I G U R E M 2 . 1 3

(for Problem M2-13)

7/29/2019 mod02



1 3

2

47

8

6

59

10

11

12

13

16

15

14

17

18

19 20

4

2

3 3

3

4

3

4

3

3

4

3

3

2

4

3

4

4

4

3

2

5

2

2

2

F I G U R E M 2 . 1 4

(for Problem M2-16)

INTERNET CASE STUDY

See our Internet home page at www.prenhall.com/render for this

additional case study: Briarcliff Electronics.

➠ CASE STUDY

United Trucking

Like many trucking operations, United Trucking got started withone truck and one owner—Judson Maclay. Judson is an individ-

ualist and has always liked to do things his way. He was a fast dri-

ver, and many people called the 800 number on the back of his

truck when he worked for Hartmann Trucking. After two years

with Hartmann and numerous calls about his bad driving,

Judson decided to go out on his own. United Trucking was the

result.

In the early days of United Trucking, Judson was the only

driver. On the back of his truck was the message: How do you

like my driving? Call 1-800-AMI-FAST. He was convinced that

some people actually tried to call the number. Soon, a number of

truck operators had the same or similar messages on the back of

their trucks. After three years of operation, Judson had 15 other

trucks and drivers working for him. He traded his driving skillsfor office and management skills. Although 1-800-AMI-FAST

was no longer visible, Judson decided to never place an 800 num-

ber on the back of any of his trucks. If someone really wanted to

complain, they could look up United Trucking in the phone

book.

Judson liked to innovate with his trucking company. He

knew that he could make more money by keeping his trucks full.

Thus he decided to institute a discount trucking service. He gave

substantial discounts to customers that would accept delivery to

the West Coast within two weeks. Customers got a great price,

and he made more money and kept his trucks full. Over time,

Judson developed steady customers that would usually have

loads to go at the discounted price. On one shipment, he had an

available capacity of 10 tons in several trucks going to the WestCoast. Ten items can be shipped at discount. The weight, profit,

and number of items available are shown in the following table:

WEIGHTITEM (TONS) PROFIT/UNIT AVAILABLE

1 1 $10 2

2 1 10 1

3 2 5 3

4 1 7 20

5 3 25 2

6 1 11 1

7 4 30 2

8 3 50 1

9 1 10 2

10 1 5 4

Discussion Questions

1. What do you recommend Judson should do?

2. If the total available capacity was 20 tons, how would this

change Judson’s decision?

7/29/2019 mod02


Bibliography M2-23

Howard, R. A. Dynamic Programming . Cambridge,MA: The MIT Press,

1960.

Ibarake, Toshihide and Yuichi Nakamura,“A Dynamic Programming

Method for Single Machine Scheduling,” European Journal of

Operations Research (July 1994): 72–82.

Idem, Fidelis and A. M. Reisman.“An Approach to Planning forPhysician Requirements in Developing Countries Using Dynamic

Programming,” Operations Research (July 1990): 607–18.

Stokes, Jeffrey et al.“Optimal Marketing of Nursery Crops from

Container-Based Production Systems,” American Journal of

Agricultural Economics (February 1997): 235–246.

BIBLIOGRAPHY

Bellman, R. E. Dynamic Programming, Princeton,NJ: Princeton

University Press, 1957.

Bourland, Karla,“Parallel-Machine Scheduling with Fractional Operator

Requirements,” IEE Transactions (September 1994): 56.

Elmaghraby, Salah, “Resource Allocation via Dynamic Programming,”

European Journal of Operations Research (January 1993): 199–215.

El-Rayes, Khaled et al.“Optimized Scheduling for Highway

Construction,” Transactions of AACE International (February 1997):

311–315.

Hillard, Michael R. et al.“Scheduling the Operation Desert Storm Airlift:

An Advanced Automated Scheduling Support System,” Interfaces

21, 1 (January–February 1992): 131–146.

Documents

mod02