IRP OK.pdf

8/14/2019 IRP OK.pdf

1/104

Inventory Routing

Problems

Martin Savelsbergh


2/104

Goals

Introduce Inventory Routing Problems

Introduce Solution Approaches forInventory Routing Problems

Introduce Inventory Routing Game


3/104

Inventory Routing

Inventory Management

Vehicle Routing

C ti l I t


4/104

Conventional Inventory

Management Customer

monitors inventory levels places orders

Vendor

manufactures/purchases product assembles order

loads vehicles

routes vehicles makes deliveries

You call We haul

P bl ith C ti l


5/104

Problems with Conventional

Inventory Management Large variation in demands on

production and transportationfacilities

workload balancing

utilization of resources unnecessary transportation

costs

urgent vs nonurgent orders setting priorities


6/104

Vendor Managed Inventory

Customer

trusts the vendor to manage

the inventory Vendor

monitors customers inventory

customers call/fax/e-mail

remote telemetry units

set levels to trigger call-in

controls inventory

replenishment & decides when to deliver

how much to deliver

how to deliverYou rely We supply


7/104

Vendor Managed Inventory

VMI transfers inventory management (and

possibly ownership) from the customer to thesupplier

VMI synchronizes the supply chain through the

process of collaborative order fulfillment


8/104

Advantages of VMI

Customer

less resources for inventory

management assurance that product will be

available when required

Vendor

more freedom in when & how to

manufacture product and make deliveries

better coordination of inventory levels atdifferent customers

better coordination of deliveries to

decrease transportation cost


9/104

Inventory Routing

Decide when to deliver to a customer

Decide how much to deliver to a customer

Decide on the delivery routes


10/104

Inventory Routing

Inventory Management

Vehicle Routing

Long-Term Problem


11/104

Inventory Costs

Transportation Costs

+


12/104

Single Customer (Single Link)

Inventory cost Transportation cost

AAA B Determine shippingpolicies that optimizethe trade-offbetween:

transp.cost

inv. cost


13/104

Problem Description

A B

Fleet of vehicles:- transportation capacity- transportation cost (ABA)

1=rc

0 1 2 3 4 5


14/104

Problem Description

A B

Volume produced in A per unit time: v Volume consumed in B per unit time: v Inventory cost per unit time: h


15/104

Goal

A B

Determine shipping policies

that minimize

inventory cost + transportation cost


16/104

The Continuous Variant

Single frequency f Continuous time between shipments t = 1/f Single vehicle

Optimal solution

vt rt 0

min hvt + ct

t

= min(p c

hv ,

r

v )


17/104

Minimum Intershipment Times

A practical constraint:Minimum intershipment times, e.g., 1 day

ZIO (Zero Inventory Ordering)

FBPS (Frequency Based Periodic Shipping)


18/104

Zero Inventory Policy

Minimum inter-shipment time Single frequency f

Continuous time between shipments t= 1/f

A shipment is performed when the inventory

level of the products is zero

*t *t

= v

v

h

vct ,,1maxmin*

Frequency Based Periodic


19/104

Frequency Based Periodic

Shipping Policies

Minimum intershipment time

One or more frequencies Integer time between consecutive shipments

0 1 2 Single frequency

Double frequency


20/104

Best Single Frequency Policy

1/k*: best single frequency

Lemma: ( )

= 1,max*1h

cvvkk

+=

vkk

chkz

kk

Best

1

SF min

0 1 2


21/104

Best Double Frequency Policy

1/k1*,1/ k2*: best frequencies

Lemma:

...k*k*k

...k*k

= T :vT(d) =PT

j=1pj(vTj(d) +S)

Probability that stockout occurs on dayj

Stockout cost

Probability that no stockout occurs

Delivery cost

vT(d) =(d) +(d)T +f(T, d)

(d)constant, f(T,d)goes to zero exponentially fast as T

(d) = pS+(1p)cPdj=1jpj

Single Customer Problem


36/104

Single Customer Problem

An optimal constant replenishment period strategy over alarge T-day planning period will correspond to choosing d*to

minimize (d)

(d) = pS+(1p)cPdj=1jpj

Expected number of daysbetween deliveries

Expected cost of a delivery

Best d-day policy


37/104

Best d-day policy

Demand: uniformly in [1,20]Deliver cost: 40Stockout cost: 50

Optimal policy


38/104

Optimal policy


39/104

Two Customer Problem


40/104

Two Customer Problem

Stochastic policy:

Storage capacity: 20

P[demand = 0] = 0.4, P[demand = 10] = 0.6

Shortage penalty: 1000 customer 1; 1005 customer 2

Vehicle capacity: 10 Individual routes: 120; Combined route: 180

Infinite horizon Markov Decision Process Minimize expected total discounted cost


41/104

Bounds


42/104

Bounds

Customer usage during period: ui

Customer storage capacity: Ci

Vehicle capacity: Q

Minimize total mileage D*

subject to

Total volume delivered to customer i ui

Maximum volume delivered per trip Q

Maximum quantity delivered to customer i min(Ci, Q)


43/104

Two Customer Analysis


44/104


1 201t

12

t

120102 ttt +=

QC >1

QC


45/104


Plant

Customer01t

12t

02t

QC 20

1

2

Assume :

Patterns:

(C1,0)

(0,Q)

(C1,Q-C1)

Extra mileage

Improved Bounds


46/104

p o ed ou ds

Delivery Patterns

Pj = (dj1,dj2, ..., djn) is feasible if

i Idji Q 0 dji Ci i I .and Pj) = {iI : dji > 0} : Customers visited inPjc(Pj) : The cost of delivery patternPj - optimal TSP value

: Set of all feasible delivery patterns

Pattern Selection LP

D* =minPj c(Pj) xjs.t. Pj dji xj ui, i I

xj

0xj : How many times

should patternPj be

used

Obstacles


47/104

Obstacle I : Infinite number of feasibledelivery patterns

Obstacle II : The calculation of the cost of

each delivery pattern involves the solution of a

traveling salesman problem

Obstacle I


48/104

Base Pattern

A feasible delivery pattern P is a base pattern

if at most one customer, say k, in (P) receives adelivery quantity less than min(Ck, Q), and, in thatcase, the delivery quantity is Q i (P)\k Ci

Theorem

The base patterns are sufficient to find an optimal

solution to the Pattern Selection LP

Number of columns of Pattern Selection LP is finite

Obstacle II


49/104

Focus on upper and lower bounds on D* instead of D* itself.

Lower bound (LBk) :If Ci < Q/k, then assume Ci = Q/k

LB1 LB2 D*If mini I(Ci) Q/k, then LBk = D*

|(P)| k for any base pattern PUpper bound (UBk) :

At most k stops in a tour

For values k=3 and

k=4, the TSPs that

have to be solved

involve at most 4 and

5 stops, respectively,

and thus can be solved

relatively easily by

enumeration

UB1 UB2 D*If Q/mini I(Ci) k, then UBk = D*|(P)| k for any base pattern P

Dominance


50/104

Do we need base patternP ?

If z c(P),

then the base patterns with j > 0 collectively dominate P

Base pattern P can be eliminated from the Pattern Selection LP

Simple Dominance


51/104

p

Consider base patternP withd4


52/104

ApproachSpecialized solver for LPs with a large ratio of

number of columns to number of rows

A Partial LP

Subset of full

set of columns

Check reduced

costs for the

remaining columns

Optimal

solution

Instance n # of patterns # of iterations default(sec) sifting(sec)

1 136 3,029,980 5 85.66 77.09

2 157 4,336,466 6 118.37 121.34

3 169 5,420,907 6 155.29 152.56

4 147 8,838,137 5 360.20 240.83

5 157 8,975,615 6 397.33 254.81

6 194 16,640,122 6 675.98 533.56

Computational Experiments


53/104

36 plants, ~2000 customers

2500000

2900000

3300000

3700000

4100000

1 2 3 4k

UB

LB

Inventory Routing


54/104

Schedule 1: 420 miles per day

10

100

10

140

3

4

100

100

100

1 2

(1000,3000,0,0),(0,0,2000,1500)

Schedule 2: 380 miles per day(0,3000,2000,0)(2000,3000,0,0), (0,0,2000,3000)

Pattern Selection LP with T=1day

Optimal Objective Value : 380

0.5 : (0,3000,2000,0)

0.5 : (2000,3000,0,0)0.5 : (0,0,2000,3000)

Pattern Selection LP found schedule 2

and it shows no better schedule exists!Q = 5000

Solution Approaches


55/104

Deterministic

Based on average product usage Stochastic

Based on probability distribution of product

usage

Deterministic SolutionApproach


56/104

Approach

Two Phase Approach Phase I: Determine which customers should

receive a delivery on each day of the planning

period and how much

Phase II: Create the precise delivery routes for

each day

Rolling horizon approach

Deterministic SolutionApproach


57/104

Approach

Two Phase ApproachPhase I: Integer program

Phase II: Insertion heuristic

Integer Program


58/104

Lower bound on the total volume that has to be delivered

to customer i by the end of day t:

Upper bound on the total volume that can be delivered

to customer i by the end of day t:

Delivery constraint:

Integer Program


59/104

Resource constraints:

Vehicle capacity

Number of vehicles

Integer Program


60/104

Vehicle capacity

Number of vehicles

Storage capacity

Integer Program


61/104

Improve the efficiency by

Route elimination

Aggregation

Insertion Heuristic


62/104

Input for next kdays:

List of customers

List of recommendeddelivery amounts

Output for next k days for each vehicle:

Start time

Sequence of deliveries

Arrival time at each customerActual delivery amount at each customer

Key Issue


63/104

How to handle variable delivery quantities?

We may be able to increase delivery amounts We may be able to decrease delivery

amounts

We may be able to postpone deliveries to

another day

Insertion Heuristic


64/104

Minimum delivery volume:

Amount suggested bythe integer program

Earliest time a delivery can be made:

Insertion Heuristic


65/104

Latest time a delivery can be made:

Maximum delivery volume:

Insertion Heuristic


66/104

For each route:

Earliest time a route can start Latest time a route can start

Earliest time a route can end

Latest time a route can end

Sum of minimum deliveries

Sum of maximum deliveries

Insertion Heuristic


67/104

Feasibility check:

Compute minimum delivery volume. Will theminimum delivery volume fit given the other

deliveries?

Compute earliest and latest delivery can takeplace. Is late greater than early?

Compute maximum delivery volume. Is

minimum less than maximum?

Delivery Volume Optimization


68/104

Observe:

The amount that can be delivered at a customer

depends on the time at which the delivery starts The time it takes to make the delivery depends on the

size of the delivery

There is a limit on the elapsed time of a route

Result:

It is nontrivial to determine, given a route, i.e., a

sequence of customer visits, what the maximumamount of product is that can be delivered on this

route !!



69/104

Tank capacity or Truck capacity

Earliest delivery time Latest delivery time

Usage rate Pump rate



70/104

There is a polynomial time algorithm that solves this

problem. The algorithm constructs a series of piecewiselinear graphs (one for each customer on the route)

representing the maximum amount of product that can

be delivered on the remainder of the route as a functionof the start time of the delivery at the customer.



71/104

Customer 1

Customer 2

Delivering a little less atCustomer 1 allows a much

larger delivery at Customer 2

Pump time +Travel time


72/104

Stochastic IRP


73/104

Determine

inventory

levels

Assign

customersto vehicles

Deliver at

customer &drive back

Load to

capacity & drive

to customer

Markov Decision ProcessModel


74/104

State, x

inventory levels at different customers Action, a

Which customers to replenish

How much to deliver at each customer

How to combine customers into vehicle routes

Objective

Solving Problems Exactly


75/104

Algorithm: Policy Iteration

For each problem

Customer capacity 10 units

Customer demand 1, , 10 w.p. 0.1 each

Vehicle capacity 5 units

Direct delivery only

MDP Model: Issues


76/104

Optimality Equation

Computing optimal value function

Computing expected value

Computing optimal action

V(x) = maxaA(x)

E[g(x, a) + V(Xt+1

)|Xt=x, A

t=a]

Approximation Methods


77/104

Idea

Approximate V with V

Motivation

Parameterized approximation function

Examples for Basis Functions


78/104

Polynomial function inventory level at customers

second order effects

4450

4650

4850

5050

5250

0 1 2 3 4 5 6 7 8 9 10

Inventory at customer 3 (X3)

Value

opt, X2 = 0

opt, X2 = 5

opt, X2 = 10

app, X2 = 0

app, X2 = 5

app, X2 = 10

Approximating the ValueFunction


79/104

Although IRP is not separable, the major

costs (including transportation) are

associated with small groups of

customers (vehicle routes)

We do not know in advance whichgroups will be in each vehicle route

We can identify subsets of customers

that can possibly be in the same vehicle

route

MDP for subset of customers


80/104

State, (xi, ki)

inventory at customers vehicleswhich could be allocated

Action, ai

deliveries to customers in the subset

Transition probability

MDPs for small subsets of customers can

be solved optimally in advance

Approximating the ValueFunction


81/104

In advance, optimally solve problems for subsets ofcustomers

On each day, partition the customers and vehicles into

subsets by solving a cardinality constrained partitioning

problem

1-customer subsets: nonlinear knapsack problem

2-customer subsets: maximum weight perfect matching

problem

Non-linear Knapsack Problem


82/104

Vehicles

Customers

0,0

2,0

1,1

2,1

1,0

0,1 0,3

2,3

1,3

0,2

1,2

2,2

0

V1(x1,0)

V1(x1,0)

V1(x1,0)

V1 (

x1 ,2

)

V2(x2,1

)

V2(x2,0)

V2(x2,0)

V3

(x3

,0)

V3 (

x3 ,2

)

0

0

V3 (x

3 ,1)

V3(x3,0)0 0

0

0

0

0

0

0

0

Computing ParametersMethod I


83/104

Objective Function

Looks like weightedleast squares

regression problem

Cannot be

computed for

large problems

Computing ParametersStochastic Approximation Algorithm


84/104

Simulate system under policy Sample path x0, x1, , xt, ...

Update coefficients

Step size

Temporal difference

Eligibility vector

t =g(xt,(xt)) + V(xt+1, rt) V(xt, rt)

Pt=0

t = ,P

t=0(t)2 <

zt+1 =zt + OrV(xt, rt)

Convergence typically very slow

Computing ParametersMethod II


85/104

Value function for policy

Looks like weighted least squares

regression problem

Cannot be

computed for

large

problems

Computing ParametersKalman Filter Algorithm


86/104

Simulate system under policy

Sample path x0, x1, , xt, ...

Update matrices Mt (similar to XX) & Yt

(similar to XY)

rt is the solution of Mtrt = Yt

Convergence significantly faster

Computing ParametersStoch App vs. Kalman F + Stoch App


87/104

0

2

4

6

8

0:00 0:09 0:18 0:28 0:37 0:46 0:56 1:05 1:14 1:24 1:33

Time (hours)

Para

meters

Customer 1

Customer 2

Customer 1

Customer 2StochasticA

pp

StochasticApp

KalmanF

Estimating Expected Value


88/104

Multi-dimensional Integral

d = #dimensions = #customers

very hard to compute Deterministic Methods

MSE = O(n2 - 2c/d)

Randomized Methods

MSE = O(1/n)

Deterministic methods are better when 2 - 2c/d < -1

Randomized methods are

better for large d

Choosing the Best Action


89/104

Based on sample averages of actions

Question

How large should the sample be so that we are reasonablysure of choosing the best action?

Nelson and Matejcik (1995)

Sample size to ensure chosen alternative has value withintolerance of best value with specified probability

Variance reduction methods

Common random numbers

Orthogonal arrays

Variance Reduction MethodsNumber of Observations for Choosing Best Action


90/104

0

400

800

1200

1600

2000

1 101 201 301 401 501 601 701 801 901Simulation Steps

NumberofOb

servations

OA

Random

Approximate Policy Iteration


91/104

1. Initialization. Simulate initial policy 0 and obtain parameters r0

2. Use parameters rt1 for policy t

and obtain actions using

3. Simulate policy t and obtain parameters rt

4. t t + 1; go to Step 2

Performance ComparisonSmall Instances


92/104

9.4

9.6

9.8

10

10.2

10.4

10.6

10.8

0 1 2 3 4 5 6 7 8 9 10

Inventory at customer 3 (X3)

Value*10-3

Optimal value

Policy 1

Policy 2

Policy 3

CBW policy

Performance ComparisonLarge Instances


93/104

Price-Direct Replenishment


94/104

A control policy based on a simple

economic mechanism for dispatching

The dispatcher receives a transfer price

diV

ifrom management for replenishing d

i

units of product at customer i.

The dispatcher is responsible for paying

the distribution costs cI, when replenishinga set of customers I.



95/104

Net value for dispatcher

Incremental value for dispatcher

PiIVidi cI

diVi (cI{i}cI)

Price-Directed Replenishment


96/104

Managements problem: Set Viso that the

dispatcher is motivated to minimize the

long-run time average replenishment costs



97/104

Management problem (single customer):

Primal Dual

min cZ

dZ =u

0 d min{C, Q}

0 Z

max uVdV c 0 d min{C, Q}

replenishment

frequencyusage



98/104

Management problem (single customer):

Dual

max uVdV c 0 d min{C, Q}

If Vis interpreted as the transferprice received by the dispatcher

for replenishing one unit, thenthis dual program maximizes therate at which transfer revenueaccumulates, subject to theconstraint that the total transfer

payment cannot exceed the coston any replenishment

Direct Replenishment


99/104

Price directed operating policy maximizing

the net value of a replenishment

max0dmin{C,Q}{Vd c}

References

P J ill t J B d L H M D (2002) D li t


100/104

P. Jaillet, J. Bard, L. Huang, M. Dror (2002). Delivery costapproximations for inventory routing problems in a rollinghorizon framework. TS 36, 292-300.

A. Campbell, M. Savelsbergh (2004). A decompositionapproach for the inventory routing problem. TS 38, 488-502.

A. Campbell, M. Savelsbergh (2004). Delivery volumeoptimization. TS 38, 210-223.

A. Kleywegt, V. Nori, M. Savelsbergh (2002). The stochasticinventory routing problem with direct deliveries. TS 36, 94-118.

A. Kleywegt, V. Nori, M. Savelsbergh (2004). Dynamic

programming approximations for a stochastic inventoryrouting problem. TS 38, 42-70.

J.-H. Song, M. Savelsbergh (2006). Performancemeasurement for inventory routing. TS. To appear.

References

L B t i M G S (2002) C ti d di t


101/104

L. Bertazzi, M.G. Speranza (2002). Continuous and discrete

shipping strategies for the single link problem. TS 36, 314-

325.

L. Bertazzi, G. Palletta, M.G. Speranza (2002). Deterministic

order-up-to level policies in an inventory routing problem. TS

36, 119-132.

L. Bertazzi, G. Palletta, M.G. Speranza (2005). Minimizing thetotal cost in an integrated vendor-managed inventory system.

JH 11, 393-419.

References

D Adelman (2003) Price directed replenishment of s bsets


102/104

D. Adelman (2003). Price-directed replenishment of subsets:methodology and its application to inventory routing. MSOM5, 348-371.

V. Gaur, M. Fisher (2004). A periodic inventory routingproblem at a supermarket chain. OR 52, 813-822.

A. Campbell, J. Hardin (2005). Vehicle minimization forperiodic deliveries. EJOR 165, 668-684.

W. Bell, L. Dalberto, M. Fisher, A. Greenfield, R, Jaikumar, P.Kedia, R. Mack, P. Prutzman (1983). Improving thedistribution of industrial gases with an online computerized

routing and scheduling optimizer. Interfaces 13, 4-23.

Inventory Routing Game

htt //k i t h d 8081/IRG
http://kronos.isye.gatech.edu:8081/IRGamehttp://kronos.isye.gatech.edu:8081/IRGame


103/104

http://kronos.isye.gatech.edu:8081/IRGame

Login: player1, , player20

Password: player1, , player20

Play Instance 3

Winner gets prize on Friday
http://kronos.isye.gatech.edu:8081/IRGamehttp://kronos.isye.gatech.edu:8081/IRGame


104/104

Questions?

Documents

IRP OK.pdf