École des Hautes Études Commerciales ... - École de gestion

École des Hautes Études Commerciales

affiliée à

l'Université de Montréal

The Two-Level Decomposition Principle Via Dual Central Pricing

for Multi-Stage Stochastic Programming

par

Lila Rasekh

HEC Montréal

Thèse présentée à la . Faculté des études supérieures

en vue de l'obtention du grade de

en administration

Décembre 2008

C) Lila Rasekli, 2008

École des Hautes Études Commerciales

affiliée à

l'Université de Montréal

Cette thèse intitulée :

The Two-Level Decomposition Principle Via Dual Central Pricing

for Multi-Stage Stochastic Programming

Présentée par

Lila Rasekh

a été évaluée par un jury composé des personnes suivantes :

Patrick Soriano, HEC Montréal, président-rapporteur

Jacques Desrosiers, HEC Montréal, directeur de recherche

Olivier Bahr', HEC Montréal, membre du jury

Shanling Li, McGill University, membre du jury

Samir Elhedhli, University of Waterloo, examinateur externe

Résumé

La programmation stochastique multi-périodes est tout à fait adéquate pour

résoudre des applications en tenant compte d'incertitudes dans le temps.

Diverses méthodes exitent pour résoudre les problèmes de programmation

stochastiques parmi lesquelles on retrouve : les approches directes comme la

méthode du simplexe et les méthodes de points intérieurs, et les approches

par décomposition comme la méthode de coupes par centres analytiques,

les décompositions de Dantzig-Wolfe et de Benders, et la méthode de forme

en L. On peut de plus aussi utiliser dans certains cas des méthodes approxi-

matives comme la simulation de Monte Carlo et les approches combinatoires

heuristiques.

En ce qui concerne la programmation stochastique à deux périodes en di-

mension finie et avec fonction de recours, les méthodes de résolution ont bien

été étudiées théoriquement et il existe des approches efficaces pour résoudre

cette classe de problèmes. La programmation stochastique à deux périodes

peut être généralisée au cas multi-périodes lorsqu'un ensemble de décisions re-

quiert une certaine évolution au cours du temps. Comme approches efficaces

de solution des problèmes de programmation stochastique multi-périodes, on

peut utiliser les méthodes de décomposition emboîtée.

Dans cette thèse, nous étudions deux aspects de la planification de la

production en présence d'une demande aléatoire : d'une part, la gestion

de la capacité, d'autre part, la production locale versus l'externalisation.

Le premier problème se formule comme une problème de programmation

stochastique multi-périodes de grande taille de façon à prendre en compte

la demande aléaloire. Pour résoudre ce type de problèmes, nous avons

implémenté une méthode de résolution par décomposition à deux niveaux

basée sur la méthode de points intérieurs ACCPM (Analytic Center Cut-

ting Plane Method), un algorithme de plans coupants calculant des centres

analytiques. La décomposition est appliquée sur le dual de la formulation

afin d'exploiter sa structure particulière. Les valeurs centrales des variables

duales obtenues par ACCPM procurent une convergence rapide en terme

du nombre d'itérations. Les résultats de cette approche sont comparés à la

méthode de génération de colonnes à deux niveaux basée sur la décomposition

de Dantzig-Wolfe.

Le second problème considère une décision stratégique au niveau de la

planification de la capacité de production d'une entreprise. La production par

des fournisseurs étrangers est souvent associé à un risque en ce qui concerne

la qualité des produits. Nous avons modélisé cet aspect à l'aide d'une for-

mulation par programmation stochastique multi-périodes de manière à pren-

dre en compte la variation de la qualité des produits fabriqués à l'étranger.

L'objectif est de déterminer le choix optimal entre, d'une part, le contrôle

de la qualité par la production interne et la sous-traitance à des entreprises

locales, d'autre part, la variation dans la qualité des produits par l'achat à

moindre coût à des fournisseurs étrangers. Nous proposons trois méthodes

de décompositon à deux niveaux pour résoudre ce type de problèmes : la

génération de colonnes appliquée à la formulation duale en utilisant AC-

CPM, la décomposition de Benders appliquée à la formulation primale en

utilisant ACCPM, et enfin, une méthode de décomposition appliquée à la

formulation duale où le premier niveau est résolu par la décomposition de

Dantzig-Wolfe alors que le second utilise ACCPM.

Abstract

Multi-stage Stochastic Programming (MSP) has many practical applications

for solving problems whose current resolution must be made while taking into

account future uncertainty. A variety of methods exist for solving stochastic

programming problems, among them are: direct methods such as simplex

and interior point methods, and decomposition methods such as ACCPM

(Analytic Center Cutting Plane Method), Dantzig-Wolfe and Benders decom-

positions, and L-shaped method. Moreover, approximation methods such as

Monte Carlo simulation, as well as combinatorial heuristics are employed to

solve this class of problems.

In terms of the solution methods, two-stage stochastic programming with

recourse and finite support is well developed theoretically and there exist

efficient algorithms to solve it. The two-stage stochastic programming model

can be extended to the multi-stage case, when a set of decisions need to evolve

over time. As efficient solution approaches, nested decomposition methods

can be implemented to solve the multi-stage case of stochastic programming.

This thesis addresses two important production planning challenges —

namely, stochastic capacity planning; and stochastic in-house production and

outsourcing planning.

The former problem formulated as a large-scale, Multi-stage Stochastic

Program (MSP) to capture the product demand's uncertainty over time.

To solve this problem we implement an innovative two-level, interior-point

decomposition algorithm based on ACCPM. This decomposition is done on

the dual of the formulation to take advantage of the special structure of the

MSP. The central prices obtained by ACCPM provides a fast convergence

and promising computational results in terms of the number of iterations.

Finally, the resulting algorithm is compared against the two-level, classical

Dantzig-Wolfe reformulation with a dynamic column generation.

The later problem addresses an important strategic decision to plan the

production capacity in business organizations. Outsourcing to overseas sup-

pliers is often associated with some risk on the quality of the products. Hence

we developed a multi-stage stochastic programming model which takes into

account the uncertainty involved with the quality of outsourced products in

the face of stochastic demand. The goal is to find the optimal choice between

in-house capacity expansion and buying from the local suppliers with assured

quality, versus that of buying from overseas suppliers. We propose three al-

ternative algorithms for solving this problem: a two-level column generation

via the ACCPM on the dual formulation, a two-level Benders' decomposition

via ACCPM on the primal formulation, and a two-level decomposition on the

dual formulation where the fist level is solved via the classical Dantzig-Wolfe

decomposition approach while the second level is solved by using the AC-

CPM.

Table of Contents

Résumé

Abstract

Table of Contents

List of Figures

List of Tables

Abbreviations

Dedication

Acknowledgements

1 Basic Definitions and Modeling Uncertainty

1.1 Introduction

1.2 Probability Spaces and Distributions

1.2.1 Decision Variables and Stochastic Parameters . 5

1.3 Modeling of a Stochastic Program 6

V

TABLE OF CONTENTS vï

1.3.1 Expected Value Program

1.3.2 Scenario Tree Analysis

1.3.3 Chance Constrained Program

1.3.4 Recourse Programs

7

8

9

10

1.4 Feasibility Sets 13

1.5 Properties of a Recourse Function 16

1.5.1 Special Recourse Structures 18 1.6 Subdifferential Characterization 20

1.6.1 The Subdifferential of a Stochastic Program 21

2 Optimization Methods for Stochastic Programming 23

2.1 Introduction 23

2.2 Simplex-based Decomposition Algorithms 24

2.2.1 The Simplex Method 24

2.2.2 The Dantzig-Wolfe Decomposition Principle 25

2.2.3 The Lagrangian Relaxation 29

2.2.4 The Benders' Decomposition Principle 31

2.3 L-shaped methods 33

2.3.1 The Single-cut L-Shaped Method 33

2.3.2 The Multi-cut L-shaped Method 38

2.4 Sampled Based L-Shaped Methods 39

2.4.1 The Stochastic Decomposition Algorithm 39

2.4.2 The Importance Sampling 42

2.5 Multi-stage Stochastic Programming 43

2.5.1 The Nested Decomposition Method 47

TABLE OF CONTENTS vii

3 A Decomposition Based on an Interior Point Algorithm

3.1 Introduction

3.2 The Analytic Center Cutting Plane Method

49

50

3.2.1 The Oracle 51

3.2.2 The Localization Set 52

3.2.3 The Analytic Center 54

3.2.4 The ACCPM and Interior Point Algorithms 57

3.2.5 Multiple Cuts 58

3.2.6 Adding and Dropping Cuts 58

3.3 The ACCPM for Stochastic Programming with Recourse . 65

3.3.1 Stochastic Programming with Recourse 66

3.3.2 The Potential Function and the Analytic Center . 68

3.3.3 ACCPM on Two-Stage Stochastic Programming . • 70

3.4 Two-level ACCPM for Three-Stage Stocha,stic Programming 71

3.4.1 Column Generation on the Dual 71

3.4.2 Bender& decomposition of the Primal 76

3.5 Conclusion 81

4 A Two-Level Interior-Point Decomposition Algorithm for Multi-

Stage Stochastic Capacity Planning and Technology Acqui-

sition 83

4.1 Introduction 83

4.2 The Technology Acquisition and Capacity Expansion Model 88

4.3 A Two-level Decomposition Algorithm 91

4.4 ACCPM 96

4.4.1 Solving the Master Problems 96

TABLE OF CONTENTS viii

4.4.2 The Two-level ACCPM 99

4.5 Computational Experiments 101

4.6 Conclusion 104

5 Solving Multi-Stage Stochastic, In-bouse Production and Out-

sourcing Planning by Two-level Decomposition 105

5.1 Introduction 105

5.2 In-buse Production/Outsourcing Planning Model 109

5.3 Solution Approaches 116

5.3.1 A Two-level Column Generation Approach 117

5.3.2 A Two-level Benders' Decomposition 123

5.3.3 The Analytic Center Cutting Plane Method 130

5.4 Computational Results 131

5.5 Conclusion 135

6 Conclusions 137

Bibliography 141

List of Figures

2.1 Scenario Tree 45

3.1 Two-level decomposition along the scenarios for a three-stage

SP 74

4.1 Two-level decomposition along the scenarios 93

5.1 In-bouse production and outsourcing planning, decision tree . 110

ix

List of Tables

4.1 Computational Results 103

5.1 Problem Sizes in the Column Generation Approach 132

5.2 Computational Results for C:AC-AC 133

5.3 Computational Results for C:DW-AC 134

5.4 Computational Results for B:AC-AC 135

X

List of Abbreviations

ACCPM: Analytic Center Cutting Plane Method. DEP: Deterministic Equivalent Program. DW: Dantzig-Wolfe Decomposition. MSP: Multi-stage Stochastic Programming. OEM: Original Equipment Manufacturing. SD: Stochastic Decomposition. SP: Stochastic Programming. SLP: Stochastic Linear Program. VOA: Vendor Operations in Asia.

xi

To the memory of my late father

and

To my beloved mother

Acknowledgements

First of ail, I would like to thank my supervisor Dr. Jacques Desrosiers for

his guidance, his confidence in my abilities and his encouragements during

my study and research. My gratitude extend to my advisory committee, Dr.

Shanling Li and Dr. Olivier Bahn for their helpful comments on my thesis.

I would also like to thank Dr. Jean-Louis Goffin for his ideas and support in

the early years of my doctoral studies.

I would like to thank my mother and my brothers for their patience,

unconditional love and encouragements. This work was not possible without

their presence. My special thanks is toward my friends Leila and Abhijit for

their support.

Chapter 1

Basic Definitions and Modeling

Uncertainty

1.1 Introduction

Stochastic programming is mathematical (i.e., linear, integer, mixed-integer

or nonlinear) programming that involves uncertainty, whereas deterministic

optimization problems are formulated with known par ameters. In most real-

world problems data cannot be collected accurately for a variety of reasons.

The first reason is due to measurement error, and the second and a more

fundamental cause is that sonne data forecast informations about the future

(for example, pro(luct demand or prices) and cannot be known with certainty.

Stochastic programming (SP) models take advantage of the fact that

probability distributions of stochastic data are known or cari be estimatecl.

The goal is to find a solution that is feasible for all (or almost ail) of the pos-

sible data instances and that minimizes the expectation of some function of

Basic Definitions and Modeling Uncertainty 9

the decisions and the random variables. Such models are formulated, solved

analytically or numerically, and analyzed to facilitate the decision-making

process. In later chapters, this dissertation discusses the ways of taking this

uncertainty into account and, specifically, illustrates how stochastic program-

ming can be used to make better decisions.

The first chapter discusses different approaches to modeling and gives

basic definitions of SP, as well as discussing two-stage stochastic program-

ming, feasibility sets and relate(' theorems. lt is concerne(' with convexity,

continuity, differentiability, and optimality conditions for the SP models and

it is concluded with a subdifferential characterization of SP models.

Chapters 2 and 3 are devoted to solution methods for SP. We catego-

rized the solution methods to SP into two types. The first type winch is

discussed in Chapter 2 is the Sintplex-based (h:composition methods, such as

the L-shaped and stochastic decomposition methods. This chapter also dis-

eusses nested decomposition as a solution method to multi-stage SP. Chanter

3 is devoted to the decomposition principles based on the Interior point algo-

rithms, such as the ACCPM (Analytic Center Cutting Plane Method). AC-

CPM vas implemented On two-stage stochastic programming [3]. In Chapter

3 we broaden the idea and propose a new decomposition method to solve the

three-stage SP initially and extend it to multi-stage SP finally.

In Chapters 4 and 5 We address two production planning problems —

namely, stochastic capacity planning, and stochastic in-house production

and outsourcing planning. We propose variants of the state-of-art two-level

decomposition algorithm to solve these problems. Chapter 6 presents the

conclusion of this dissertation and describes sonie unsolved problems.


1.2 Probability Spaces and Distributions

In real-world applications several parameters of a problem can be uncertain

and hence appear as a random occurrence. Future sales of a particular brand

depend on market. conditions. The replenishing and reordering point of an

inventory depends on the price changes in the market. The political and eco-

nomical situation in Asia, may affect the price of some options in the North

American market. Due to the recent strategy of globalization adopted by

many corporations, nowadays modeling uncertainty is an indispensable part

of decision mak-ing on the operational management level. Outsourcing raw

materials, labor and finished goods from so-called low cost countries to the

developed part of the world introduces mai' parameters of uncertainty to the

decision-making process. In contrast, stochastic programming provides es-

sential modeling and solution techniques to reach the optimal solution under

uncertain conditions.

Since stochastic progranuning deals with random data, we need to in-

trochice sonne notation. Letter is used to denote a random variable, w

represents a possible realization of out of the set of ah l possible outconnes

Q. The outcomes may be combined into a subset of Q called random events.

A is a collection of random events (set of subsets of Q). To each event A e A is associated a probability P(A); the triplet (Q, A, P) is called a probability

space.

A random variable cari be discrete or continuous. It is a dis crete van dom

variable if it eau assume onlya finite or countably infinite number of val-

ues. On the other land, it is a continuons vandom variable if it cari assume

any values in an interval of real numbers. A list of possible values of a dis-

Basic Defin,itions and Modeling Uncertainty 4

erete random variable, together with their assoeiated probabilities give us

a discrete probability distribution. The probability of a continuous random

variable is flot defined at specific values. Instead, it is defined over an inter-

val of values and is represented by the area !limier a eurve. For a particular

random variable e, the cumulative distribution is defined as

Fe(x) = x),

or more precisely

Fe (x) = P < .

Continuous random variables cari be deseribed through their density function

f() iffhere

P(a < b) = f (6,- )ck. (1.1)

The expeetation of a random variable is computed as

E[e] = > f () for discrete random variable ; kER

or equivalently

E [ ] f+oc œ dk-'() for continuons random variable.

The expectation of er is ealled the rth moment of e and denoted by =

Basic Dejïnitions and Modeling Uncertainty

1.2.1 Decision Variables and Stochastic Parameters

Stochastic programming deals with mathematical programming models where

some parameters are "incertain. For stochastic programming, some variables

are to be set by a decision-maker (these are called the decision variables),

while soine model parameters are the random variables. There are a variety

of situations one might consider in tins context. One differentiation is based

on when the deeision maker must make decisions relative to the time when

the random variables are realized. There are several possibilities winch are

described as follows:

No Recourse Model: The decision maker must choose values for the

deeision variables before any of the random variables are realized.

Single Stage Recourse: Some of the decision variables must be set

before the random variables are realized, while others may wait until they

are realized.

Multistage Recourse: Decisions and random variables are determined

in stages. The first set of decision variables must be fixed before ail realiza-

tions. Then the first set of random variables is realized and the second set of

decision variables must be fixed. The process continues through a series of

decisions and realizations. Typically, the stages represent intervals of time.

Wait and See: The decision maker makes no decisions until ail random

variables are realized. In the wait and see approach, the optimal decisions cari

be made with no uncertainty using deterministic optimization algorithms.

Among the above possibilities, the no recourse case is the most difficult

because all the decisions must be made before the model is completely known.

whereas decision making under the wait and sec paradigm is tire easiest.


1.3 Modeling of a Stochastic Program

This section discusses the Stochastic Linear Program (SLP) and its Doter-

ministic Equivalent Program (DEP). DEP is a large scale linear programming

problem with ail scenarios and constraints, winch is used in many real-life

applications.

In general, stochastic linear programming consists of

min z = ex

s.t. As = b, (1.2)

Tk x > hk , k = 1, . K,

> O.

where the deterministic constraints are represented by As = b (a; is a decision

variable) and the stochastic constraints are formed by

Tics > hk , k = 1, K.

The real value of (Tk , hk), k = 1,. K is not known, but the probability dis-

tribution of (T, h) is assume(' to be defined. Hence, the existing uncertainty

is expressed by probability distributions. i.e.,

P { (T, = (Tk. hk)} = pk, k= 1, . . K.

In most cases a stochastic programming model decides on s here-and-now

without knowing the value of (T, h) but knowing its probability distribution.


The stochastic constraints, Ti > h (we drop index k for simplicity), have

to be replaced by sonie sort of deterrninistic equivalent constraints to be

able to solve the SP model. These stochastic constraints are interpreted

as goal construiras. There are a variety of approaches that one might take

for replacing goal constraints by their deterministic equivalent, among them

expected value, scenario tme, chance construiras and recourse problems are

explained next.

1.3.1 Expected Value Program

If T and h follow discrete distributions, TX > h can lie replaced by Ta; > h where

T = E pkTk and h =)khk

where T and h are expected values of T and h respectively. Replacing the

probabilistic ternis by their expected values 'flakes the model deterministic.

This approach models the no-recourse situation.

Expected value solution serves the solution to this problem. However,

the resulting solution may be 'instable in the sense that it might perform

poorly under realization of some random variables while performing well

under others. In this case, the optimality may be sacrificed to obtain a

robust solution, which is suboptimal in terms of expected value, but lias a

lower risk. This approach is simple and easy to compute.


1.3.2 Scenario Tree Analysis

In many real-life applications, the distribution of a random variable is not

known, or it is too expensive to consider a discrete distribution with many

possible outcomes, or to handle a continuons distribution with numerical

integration. in such cases a question arises whether one cari select a small

munber of realizations called scenarios to represent the random events. in

strategic planning, scenarios obtained through expert judgments and some

of the realizations of random events are considered in detail; in other cases,

scenarios may be a quintile of a kno-wn distribution or historical data.

One way to deal with such difficulties concretely is the scenario tree anal-

ysis. In this approa.ch each scenario is given a probability. The scenario

informations and their probabilities are then organized in a tree structure.

Scenario trees provide fiexibility to choose scenarios which need to be con-

sidered and their relative importance. But it is impractical to consider too

many scenarios, and this limas the level of detail of the model. It is common

to use scenario trees in financial planning models. Sirice it is (-casier to predict

trends in the near future, the scenario tree in such models lias more branches

at each node of the earlier stages and fewer towards the end of the planning

period.

A simple solution method to scenario analysis woukl be solving the SLP

in (1.2) for each scenario separately. The proposed optimal solution to SLP

would then be the average or the sweighted average of solutions of ah l sce-

narios.

The scenario tree imposes limitations to modeling by not including ail

possible scenarios in the model. However, it provides a flexible approach to

Basic Definitions and Modehng Uncertainty 9

modeling problems with stochastic parameters, especially t'ose in winch the

random data forecast some events in the future (for example, the price of an

option in the market), in which the distribution of random parameters is flot

known in advance.

1.3.3 Chance Constrained Program

Chance construira programming is mathematical programming in which some

constraints are stocha.stic. In model (1.2) chance constraint replaces Tx > h

by

P{Tx > > a, (1.3)

where a E [0,1]. This can be interpreted, for example, as requiring the

power supply system to be able to cope with customer demand at least 9,5%

of the time. This formulation is mostly used when it is impossible or too

expensive to guarantee that the conditions eau be held at ail times and the

cost of failing those conditions is dillicult to quantify. In the power supply

example, equipments may fail but it is important to keep the failure at a

low level to avoid the high cost of power shut clown. A chance constrained

program is often converted to a deterministic equivalent to be solvable. The

model is taking care of risk, (i.e.. 1 — a is the maximum acceptable risk)

but, it may introduce non-convexity into the problem. If the probability

distribution of the random parameter satisfies some convexity requirements,

then the feasible region is convex and closed, and techniques of mathematical

programming cari be used to solve the problems. In chance constrained

programming [13], the objective is an expectational function (the E-model)

Basic Definitions and Modeling Uneertainty 10

or it cari be the variance of some result (the V-model), or the probability of

some occurrence (the P-mode!). Another type would be the objective that

is a: quintile of a distribution function [44]. For a summary of properties and

solution methods to chance constrained programs, sec for example [10].

1.3.4 Recourse Programs

Stochastic programming with recourse is a general purpose technique that

can deal with uncertainty in any of the model parameters. The simplest

model of recourse problems have two stages:

In the first, stage, the decision maker makes a decision. In the second

stage, the decision maker observes a realization of the stochastic elements of

the problem but is allowed to make further decisions to avoid the constraints

of the problem becoming infeasible.

This model introduces corrective (recourse) actions. It replaces

> h by TX+Wy = h,

where y is a decision vector on the second-stage problem. The choice of y is

dependant on the realization of (T, h) in the first-stage. The model associates

the cost q to be added to the objective function for acquiring information


and taking a recourse action. Therefore the recourse model is

miii CT X qkPkYk

S. t. Ax = b.

'Tk x + .147yk = hk,

Yk > 0, x > O.

k = 1, . (1.4)

This model takes care of the risk effectively but the problem may get large

in ternis of the constraints and variables very easily.

As it is mentioned above, there is a distinction between the first stage and

the second stage. The first stage is represented by vector x and corresponding

first-stage vectors and matrix A .b, c of sizes in x n, ru x 1 and n x 1. In the

second stage, a number of random events w E may be realized. For a

given realization of w, the second-stage problem data q (w) ,T (w), and h (w)

become known where q (w) is k x 1, h (w) is I x 1, and T (w) is I x n, W is called

the recourse matrix of size I x k. In most cases we consider W to be fixed.

It allows us to define the feasibility region winch facilitates computations.

Each component of q(w),T(w) and h(w) is a possible random variable. Let

Tj (w) be ith row of T (w). Putting together the stochastic components of

the second-stage, one obtains the random vector

(w) = (qT (w ) 112- (W) 211 (W) ; • • • Trn2 (W))

Thus the cla,ssical two-stage stochastic programming with fixed recourse

Basie Definitions and Modeling Uneertainty 12

can be written as

min cTx + E [qT (w) y (w)]

s.t. Ax = h, (1.5)

T x 11Ty (w) -= h (w)

y (w) >0, x > 0.

Let

Q (x (w)) = min {qT (À)) y Wy = h (w) — T (w) x y > 01 (1.6)

be the second-stage value fUnction, and define the expected second-stage

value function as

(r) (1.7)

Then the deterministic equivalent program, (DEI') to (1.5) would be

min cTx + Q (x)

s.t. A.E = b, (1.8)

> 0.

On the other hand the recourse function to a stochastic programming prob-

lem with fixed recourse (W is deterministic) and fixed T, is represented by

(w)) = min {qT (w) y Wy = h (w) — x, y >

W (x) = (x ,(w))1, (1.9)

Basic Definitions and Modeliny Uncertainty 13

where the deterministic equivalent program (DEP) with the sanie assump-

tions would be defined as

min c7'3: + q (X)

s.t. Ax = h,

Tx —X = 0,

> 0.

(1.10)

1.4 Feasibility Sets

The recourse structure can be fixed or deterministic. For most problems we

try to restrict ourselves to the fixed recourse. Since studying the fea:sibility of

the second-stage solution depends on whether W is fixed, here, we consider

the general case, where W is a random matrix, by that the recourse function

would be

Q (w)) = min { (w) y I W (w) y = h (w) — T (w) ,x , y >

Q (x) = E [Q (x,(c..,;))1. (1.11)

Let us define the feasibility set for the deterministic constraints as K IL =

{x Ax = h, x > 0} and K2 = {x Q(X) < Do} be the second- stage feasibil-

ity set. We may then define DEP as

min cTx Q (x)

E K1 n K2. (1.12)


Definition 1.1 C R.n is the support of 4", where it is the smallest close

subset in R," such that P E = 1.

The definition of K9 is flot very helpful since we often would like to know

in advance if the second stage solution is limite for a particular z. Therefore,

we define elementary feasibility sets as an alternative &finition

K2 () 7= {x I (2(3 . < +00 . (1.13)

The possibility inteipretation of second-stage feasibility sets also defines as

follows

{x V e Q. 3 y > 0, s.t. Wy = h — Tx} = neest 11.-2((„q

The decision vector x belongs to 4 if the a feasible second-stage solution y

can be taken for ail values of The following theorems clarify more details

about the feasibility and convexity of the feasibility set in the second-stage.

Vve may skip the proofs and refer the reader to [10] for more details.

Theorem 1.1: I. For eaeh the elementary feasibility set is a closed

conyez polyhedron, hence the set .1.q) is closed and convez. II. When Q is

finite, I then is also polyhednd and coin cides with K2.

Proposition 1.2: If lias finite second moments, then

P C2(x,) < oo) = 1 implies Q(z) < o


Theorem 1.3: For a stochastic program with fixed recourse where has

finite second moments, sets K2 and Kr coincide.

Theorem 1.4: When W is fixed and has fuite second moments:

((2) K2 is closed and convex.

(b) If T is fixed, K2 is polyhedral.

(c) Let QI- be the s'apport of the distribution of T. If h() and T() are in-

dependent and Ç-1),7-, is polyhedral, then K2 iS .polyhedral.

Theorem 1.5: For a stochastic program with fixed recourse, Q (x,) is:

(a) a piecewise linear convex function in (h. T);

(b) a piecewise linear concave function in g;

(c) a piecewise lin car convex function in x for ail x in K = K1 n K2.

Definition 1.2: 4"has absolutely continuous distribution F() if there

exists a density function such that

Fe (l') = fe(t)dt.

Definition 1.3: A function g :R. ---> R. satisfies a Lipschitz condition

with d is a metric and a constant C if

d(g(ti ), g(t2 )) < C d(ti .t2 ).

A function that satisfies the Lipchitz condition for fuite C is called Lipchitz

continuons, which is a stronger condition than regular continuity.

Basic Definitions and Modeliny Uncertainty 16

Theorem 1.6: For a stochastic program 'with fixed recourse where lias

fuite second moments:

(a) Q(3.:) is a Lipschitzian convex lanction and is fuite on K2.

(b) When is finite, Q(x) is piecewise linear.

(c) If F(C) is an absolutely continuons distribution. Q(x) is differentiable on

K2.

From the above theorems, we learned that if e cornes from a fuite dis-

tribution, then 14 is polyhedral and (2(x) is piecewise linear and convex in

K. Or if e cornes from continuons distribution with fuite second moments

then (2(x) is differentiable and convex.

1.5 Properties of a Recourse Function

Recourse models are the most commonly used class of models in stochastic

programming. In many SP models, it is not possible to require that ail

constraints be satisfied for realization of ail random variables. Therefore, the

decision maker may either require the satisfaction of some constraints with

given probability, or the incorporation of a corrective action (recourse action)

into the model, when a constraint is violated. Recourse models incorporate

the corrective action to overcome the stochastic infeasibility of the decision

variables. However, in many recourse modeling situations we need to cho ose

°n'y a few scenarios to be built into the model. These choices are based on

the penalty cost winch one may be willing to pay for taking the recourse

action for a particular constraint.

The deterministic equivalent program (DEP) to the recourse model is


represented by

min fc + EK2(x,41 = min {c + (2(x) Ax = xe 'A x=b xER4 1:Ç

where Q(x..(w)) in (1.6) is a. minimization problem involving, say In goal

constraints in the form of W y = h(w) — T(w)x. In cases where it is too

expensive or simply impossible to include all goal constraints in the model, we

have to be selective about the goal constraints winch are to be incorporated

in the objective function. To make this choice, the following recourse model

which gives a clear interpretation of goal constraints is often used

min {cTx + E[v(h(w) — T (44)1 zEnT4:Ax=1)

= n'Un {cTx + E[v(z)] I T(w)x + z = 14» , (1.14) xETtlft

where z is n vector of deviation froin the goals, with components z = h i —

ta;, and v(z) is a penalty cost lanction for the deviation of z. 1f the linear

programming duality holds, v(z) eau also lie interpreted as

v (z) = min {qty I Wy = = max{z t I W`t, < (1.15) yETZT4.' tETC

The recourse function y gives the minimum recourse cost to compensate for

deviation z in the goal constraints. In a simple application of the two-stage

recourse problem this minimum cost to take a corrective action helps to

decide on the combination of scenarios winch are to be built into the model.


1.5.1 Special Recourse Structures

For sorne value of z the recourse penalty function v, given in (1.15), ma

get the value v(z) = +oc where the DEP is infeasible; or it is possible that

for some z, the value v(z) = —oo, then DEP is unbounded. In the former

situation there may exist deviations h(w) — T(w)x that cannot be corrected

in the second stage. In the latter case however, there might be a deviation

winch is extremely advantageous where the penalization for the deviation

is not possible. To avoid such situations the following designs of recourse

structure are made.

Definition 1.4: A stochastic program lias relatively complete reeourse

if Ki, C K2 i.e., every solution x that satisfies the first-period constraints,

Ax = b. has a feasible completion in the second stage. Equivalently it means

that v(h(w) — T(w)x) < oo for ail w e Q and X C C X I AX =- . Definition 1.5: A stochastic prognun lias complete recourse if there ex-

ists y > 0 such that Wy = z (z = h — Ta:) for ail z E R.m. This implies that

for ail x ,T (w), h(w), we have Q(x, w) < oo. Complete recourse is a property

of W. 1f the columns of W spans 7"?,' then Vz e R_ : Wy = z, and we

have complete recourse.

There is a special case where the penalty cost q is separable in torils of

surpluses and shortages. For example, this can be interpreted as a production

planning, where the demand is a non-deterministic parameter. Modeling the

problem as a two-stage stochastic programming with recourse ; in the first-


stage, the mode decides on the quantity of the production. However, in the

second-stage the recourse action is taken, i.e., if there is a shortage of supply,

it is needed to satisfy the demand from other sources at the penalty cost of

whereas the surplus of supply eau be inventoried at the cost of a - .

Suppose that the penalty costs q+, a- , corresponding to second-stage

variables y+ and y- , as defined above are given, the general SP model can

be reformulated as

min cT r ± Emin {q+ (w) y+ + g- (w) y- (4]

s.t. = h. (1.16)

T (w)z + W (y+((.4 + ?[(w)) = h (4 ,

Y± (w) O. Y- (w) > O, z > O.

Definition 1.6: A stochastic program lias simple recourse if

1 0 • 0 -1 0 • 0 \

0 1 • • 0 0 -1 0

W = [I, -.I"] (1.17)

0 0 1 0 0 -1

and P > = 1. Here we construct W so that it spans

For a simple recourse problem with W = [I, - I], y and q are divided as


(y+, y- ) , (g+ q- ), and the second-stage model changes to

min E [g+ (w) y+ (w) + q (w) y- (w)]

y+ — y- = (w) — (w) = 1, rn,

y>0y>O,e> O.

Theorem 1.7: Suppose the two-stage stochastic program is feasible and

has simple recourse and that (e, has finite second moments. Then (2(x) is

finite if and only if qi+ (.17 > 0 with probability one.

Above definitions and results can be used to obtain a specific optimality

condition for SP, sec for example, [22, 74. 75].

1.6 Subdifferential Characterizat ion

Marly solution methods to stochastic programming problems are dependant

on the description of subdifferentials of the model. In problems with com-

plete recourse ancl fuite support, Q(x) is convex and piecewise linear. These

properties help to build the subdifferential of Q. The characterization of this

subditferential is the topic of this section.

Definition 1.6 A vector g E is a subgradient of f : 1Zn at

E domf if for ail y E dom f, f (y) > f(x) + gT —

If f is convex and differentiable, then its gradient at z is a subgradient.

But the subgradient exists even if f is net differentiable at z. There can also


be more than one subgradient of the function f at point x. In other words a

vector g is a subgra.dient of f at z if the affine function f (x) + (y — is

a global underestimator of f. A function f is called subdifferentiable at z if

there exist at lea.st one subgradient at z. The set of subgradients of f at point

is called the subdifferential of f at z and is denoted by f (i;). A function

f is called subdifferentiable if it is subdifferentiable for ail z E dom f

Subgradient of a differentiable function

1ff is differentiable and convex at z , then ôf(z) = {77 f (x)}, i.e., its gradient

is its only subgradient. Conversely, if f is convex and Of(x) = {g}, then f

is differentiable at z and g = V f (x).

The minimum of a nondifferentiable function

A point z* is a minimizer of a convex function f if and only if it is subdif-

ferentiable at z* and 0 E 0 f (x*), i.e., g = 0 is a subgraffient of f at z*. This

condition 0 e 0 f (x*) reduces to (x*) = 0 if f is differentiable at z.

1.6.1 The Subdifferential of a Stochastic Program

Stochastic programming is a class of nondifferentiable optimization problems.

In the case of complete recourse, (2(x) is a finite convex function. Moreover,

it is â piecewise linear function if the distribution of w is discrete with finite

support. To obtain the subgradient of (2(x) let us redefine it as Q (z) =

E[v (h(w) — T(w)x)]. Consider z to be muai to h(w) — T(w)x, then v(z) cari

be interpreted in terms of the extreme points of the dual feasible region. Let


A = {Al; A2 A1 1 1} be a set of extreme points of ft e R.tm I tliTt <

Since cadi of those extreme points Ak. k = 1, . .14 is a potential optimal

solution to (1.15), v(z) cari be represented as

v(z) = inax {zt Ak} z E 7Z7n. k=1,...,Ini (1.18)

Therefore, v is a pointwise maximum of a finite set of linear functions of z.

Furthermore suppose Li hm a discrete distribution, say P{w = LiS} = ps ,

s =1, . , S. then the convex function

Q(x) psv(h.(ws) — sES

is piecewise linear in :c. In cadi piece one lias Q (x) = EsEs ps Ask (h(ws) — T(ws)x).

h follows that

OQ (x) = P.sAZ ( — T(ws )) (1.19) scs

Iii general, one can derive a formula for the subdifferential of Q at x by

using a. chain nue:

0Q(z) =E[ax — T(w)x)] = A*T(4], (1.20)

where À* is any optimal solution to max,,, E7z.{A(h(w) — T(w)x) AH/ < (IL

Subgradient inequalities and subdifferentials are used in many decompo-

sition techniques. These techniques are discussed in the following Chapters

2 and 3.

Chapter 2

Optimization Methods for

Stochastic Programming

2.1 Introduction

Marly stochastic prograniming problems have very large number of scenarios

and special structures which are tractable by decomposition. Decomposition

methods are ways of solving a sequence of optimization problems whose sce-

nario sets var y increasingly in size based on the information feedback given

by the algorithm. Hence, the decomposition algorithms recluce the size of

the problem to make it solvable. The classical decomposition methods for

mathematical programming trace back to Dantzig-Wolfe and Benders. These

decomposition principles are translated into L-shaped method in stochastic

linear programming [72]. L-shaped method lias been extended to the nested

decomposition method and developed as a computer code [29] for linear nml-

tistage recourse problems.

Optimization Methods for Stochastic Programming 94

The L-shaped method is viewed as a simplex-based &composition prin-

ciple in contrast to the method whieh is based on interior point algorithm

such as the Analytie Center Cutting Plane Method (ACCP11 ,1), sec [34, 33].

ACCPM lias been implemented as a &composition method for two-stage

stochastic programming [3]. The goal here is to apply tins method to multi-

stage stochastic programming as a nested decomposition algorithm.

2.2 Simplex-based Decomposition Algorithms

2.2.1 The Simplex Method

The simplex method considers the standard LP problem

min CTX

S.t. AX = h, x > 0,

where e E R.n , A E lZmx" and b G TC . It uses the filet that n — rn variables

must be zero at a vertex solution to partition the variables into basic, x > 0,

and non-basic variables x i = O. Similarly, A and e are partitioned as: A =

[BI AT], e = (eB, eN ). The basic variables and objective ftinction in ternis of

xN are expressed as

xB = NxN ), T CTX = CBXB CNXN = CT B —l b (CN — NT B -1 cB)xN


If any component i of c/N = CN - NTB-l cB (the reduced cost vector) is

negative, then the objective function value cari be decreased by letting x ,

become positive. i.e., basic. The change in x 8 then would be B-1 (b- Nixivi )• Variable xN, is increased until one of the basic variables becomes zero and

non-basic. The main computational cost of the simplex rnethod is calculating

B-1 cB and B-1 Ni . This can be clone by using QR or LU factorizations of B

and updating factors at each iteration. The simplex method moves between

adjacent vertices untill no improvement can be made. If the simplex steps

are non-degenerate, then the algorithm terminates after a finite number of

iterations.

2.2.2 The Dantzig-Wolfe Decomposition Principle

The revolutionary paper by Dantzig-Wolfe in 1960 [19] introduced a new

venue in solving a large-scale linear programming problems. In practice the

Dantzig-Wolfe decomposition method is most effective when the coefficient

matrix of a linear program have a block angular structure. The independent

blocks of such matrices are linked by coupling or complicating constraints.

This can be interpreted, for example, as requiring the production of several

items in a single machine with limited capacity (capacitated lot-sizing prob-

lem [21]) and a finite horizon. The objective function of such a problem

minimizes the total cost, consisting of the set up cost, the variable produc-

tion cost, and the inventory holding cost. Moreover, the constraints repre-

sent the inventory balancing constraints in each period, the set up forcing

constraints, and the capacity constraints. In the absence of the capacity

constraints, this particular problem decomposes into an uncapacitated lot-

=1, Ai > 0, = 1, ... k, i= 1

Optimization .illethods for Stochastic Programming 96

sizing problem for each single item. The independent block of the coefficient

matrix emerges from the set up and inventory balancing constra.ints whereas

the capacity constraints are regarde(' as the coupling or complicating con-

straints. In Dantzig-Wolfe decomposition the capacity constraints are kept

in the master problem. The linear program is then solved for each item winch

is regarded as a subproblem. Therefore the general idea behind this decom-

position principle is to keep the linking structure in the master program level

and to address the subproblems independently in order to take advantage of

their -special structure.

hi the next paragraphs, we study the Dantzig-Wolfe reformulation along

with the column generation technique winch leads to the decomposition al-

uorithm.

Consider the linear program

min z = cTx

s.t. (2.1)

To simplify the discussion, assume the feasible set X = {xl D x > d, x >

is a bounded convex polyhedron. Therefore, any E X eau be represented

as a convex combination of the extreme points of this polyhedron [55], i.e.,

where xi , j = 1 k, are the extreme points of X. Note that the original

Optimization Methods for Stoehastic Programming 97

linear programming problem (2.1) eau be represented by d'ose elements of

X which satisfy the set of inequalities Ax > b and minimize z. Therefore by

substituting x = Àx into the constraint set Ax > b, problem (2.1) eau

be re-expressed in terms of À as

min z = c Aixi) À i=1

s.t. Aixi > b, i=1

E = LÀ >O. i=1

Defining Axi = pi CT x = A the original problem is reformulated as

min fi Ai

s.t. (2.2)

i=1

The previous program is called the master problem winch lias fewer con-

straints but a larger number of variables than the original problem. The

constraint = 1 forms a convexity row for the n'aster program.

When the simplex method is applied to this master problem instead of

considering ail the columns we use the column generation technique, gener-

ating colunms to enter the basis as needed. To observe how this is done,

let (7r, 7r0) be the set of dual multipliers associated with the constraints


Eik_i pi > A i = 1 ,respectively in (2.2). In order to choose the vari-

able As to enter the basis, the simplex criterion asks us to find the minimum

negative reduced cost, i.e., — ps — } winch by the &finition of

p„ and f, muais to the mins {(cT — 7r 9 A)x8 — 701.

Sirice xi is an extreme point of X and the optimal solution of a linear

program whose feasible set is bounded takes place at the extreme points of the

bounded polyhedron, then finding the minimum reduced cost is equivalent

to solving the subproblem

min ((T — z A)3; — (2.3)

s.t. D x > (i.x > O.

Hence by solving (2.3) and obtaining the solution xs , one can find the

column ps = (Axs , 1) T to enter the basis of the master program. The cost

coefficient for this variable then would be = cTx s .

As mentioned above, tins method is most efficient when the coefficient

matrix of the original problem lias a block angular structure. Suppose that

the coefficient matrix and the right hand sicle of the constraint equations in

(2.1) are partitioned into

A2 • Ani . b=[b] .

and

D= D2

D771

b=

(11

(12

(Lm


where D has a block angular structure. Then the subproblem (2.3) becomes

ni

min (CiT - 7F Ai):Ci - 7F0

i=1

s.t. Dx > d , i =- 1 in (2.4)

Since the objective function and constraints in (2.4) are separable in ternis of

1„ in, the resulting subproblem is split into ni independent smaller

problems:

min (ciT — 7F Ai):Ci - 7r0

S.t . Dx >

xi > O.

As a result of this disag regation the master program now contains ni

convexity rows. In this case there exist varions strategies as how to add

columns to the master problem. One simple possibility is to solve ail the

subproblems and to add a column to the n'aster problem for cadi subproblem

with negative reduced cost.

2.2.3 The Lagrangian Relaxation

As ‘ve have seen in the capacitated lot sizing problem in the last section,

many optimization problems can be modeled as an casier program in the

absence of its complicating constraints. One way to deal with this type of

constraints would be to replace them in the objective function with a penalty

Optimization Methods for Stochastic Progiumming 30

terni. Lagrangian relaxation incorporates the violation of the complicating

constraints in the objective function and penalizes it by the weight of its dual

multiplier [54]. The resulting lagrangian problem provides a lower bound on

the objective value (for a minimization problem). The lagra.ngian relaxation

is closely related to the Dantzig-Wolfe decomposition. It eau also be donc in

the dual for complicating variables which is the topic of the next section and

was first developed by Benders.

To formulate the lagrangian problem consider the following LP

min cTx

s. t. > b, (2.5)

Dx > d,

x > 0.

Suppose that the problem is complicated the constraints Ax > b. To

transform this problem to the relatively casier one, we define a set of multi-

pliers u > 0 and add the negative tenu, —u(Ax—b), to the objective function

min cTx — u(Ax — b) (2.6)

D x > d, X > 0, u > 0.

By choosing u as a non-negative variable it is clear that the optimal value

of the lagrangian problem is a lower bound on LP. However, at optimality

the optimal value of the lagrangian problem is equal to that of LP. This

happens due to the fact that at the pair of (e, u*), primal-chiai optimal

values, u* (Az* — 0. This is known as the complementary slackness


conditions.

The lagrangian method iteratively solves a sequence of relaxed programs

as it appeared in (2.6). In each iteration, the vector of multipliers u is

computed, sec for example [27] for details on computing u. The most popular

method however to compute 'IL is based on subgradient methods [27], but it

can be shown that lagTa.ngian relaxation is in fact totally equivalent to the

Dantzig-Wolfe decomposition (again sec [54]).

2.2.4 The Benders' Decomposition Principle

The Benders' Decomposition principle is based on a variable partitioning pro-

cedure for large-scale mathernatical programming problems [4]. The method

partitions the variables by fixing the so-called complicating variables. This

reduces the given problem to a more tractable hence casier problem to solve.

The later problem is parameterized in ternis of the complicating variables

vector. The dual to this problem forms the subproblem. The subproblem

passes forward the current dual solution (for a fixed vector of complicating

variables) to a n'aster program in the form of a Benders' eut utilizing the

linear programming duality to reflue the value of the complicating variables

vector in cadi iteration.

To illustrate the Benders' decomposition method consider the following

linear programming problem:

min cT x+fT y

s.t. Ax + By > h, (2.7)

y E Y,x > O.


If y = y is fixed the resulting LP model to solve becomes

min (Ta. + fT

S.t. Ax > b — By,x > o.

Therefore, the original problem is equivalent to

min[f Ty + min{cT xlAx > b — BO]. yEY x>0

The dual to (2.8) is

max. ( 1) — BOL +

s.t. Anu, < c, iL > O.

Utilizing the linear programming duality, this program provides a lower

bound to the original problem, i.e., for cadi solution u i*, cTx > (b — R)ui* +

ry. Note that the dual feasibility is independent of the value g ; hence the

inequality crx > — By)n7 + fT y is valid for ail y. liherefore, each time a

new y is considered, the valid eut can be added to the master program where

alter I iterations the master problem becomes

min z

s.t. > (b — By)u: + fT y, i =

C

In each iteration i, the master program solves for y:, next the optimal dual

value Ît; is needed to be obtained from (2.9), assuming the dual is bounded.

(2.8)

(2.9)


(In cases of unbounded dual, another category of Benders cuts can be added,

sec for example [55].) Finally the corresponding Benders' eut is added to the

n'aster program. This procedure stops when the distance between the upper

bound and the lower bound zi (i.e. jz — I) fails below a predetermined

threshold.

By solving the Benders' master program and the subproblem ; it is possi-

ble to generate a sequence of lower and upper bounds on the original problem.

Since the master problem is a relaxation of the original program (we assume

a minimization problem here), it generates a lower bound, whereas the sub-

problem discussed earlier provides the upper bound on the original progTam.

Benders' decomposition technique is translated to L-shaped method in linear

stochastic programming (SLP).

2.3 L-shaped methods

2.3.1 The Single-cut L-Shaped Method

The basic idea behind the L-shaped method is to approximate the recourse

function C2(x) in the objective of (1.8). The recourse function Q(x) is the

expected value function in the two-stage model. It involves a solution to ail

second stage recourse linear programs and its approximate value is repre-

sented by the single variable O. The optimal value of 0 is then determined

iteratively by using the LP duality to construct a piecewise linear approxima-

tion to Q(x). The Benders' cuts which are generated to obtain the optimal

value are called the feasibility and optimality cuts.

To make this approach possible, consider a two-stage SP model with finite

Optimization Methods for Stochastic Prognenming 34

support. The two-stage recourse formulation in (1.5)-(1.6) eau Le reformu-

lated as • min cTx + .T + PetT

Y2 + • • + Psq Ys

s. t. Ax =h,

+1-4/Yi

T2x + W y2 =h2, (2.10)

nx + Wy, =hs ,

X E X, Yi, • • Ys e

where y, = y (w,), is the recourse action to take if scenario w, °ceins. The

above matrix of the linear program bas a dual-block-angular structure which

bas led to the naine L-shaped method. The method is basically a Dantzig-

Wolfe decomposition [191 of the dual formulation or a Benders' &composition

[4] of the primai formulation.

Algorithm 2.1: The Single-cut L-Shaped Method

Step 0 Set r k, y =-- 0

Step 1 Set V = y + 1 and for (xi', solve the following linear program:

min eTx + 0

s.t Ax = h,

Do; > dt , = 1 r .

0 > + Etx, = 1, 2,

> 0, 0 E R,.

(2.11)

Optimization Methods for Stochastie Programming 35

Step 2 For s = 1 S, solve the second-stage feasibility problem:

min w = eTv + + .

s.t.

,v+, 1) > O.

Wy v+ — = lis

s

— Ts xv, (2.12)

If w > 0, let u be a dual multiplier and set r = r 1. Set Dr = o-TTs , dr = uT hs and return to step 1;

else w = 0 for ail s = 1, • • S, go to step 3.

Step 3 For s = 1. • • , S solve the second-stage problem:

Qs (xv) = miii

s.t Wy = h — Tsx L . (2.13)

y

Let the dual multiplier be

Set k = k + 1. Ek = — Es s T p Es p h,. s=1 s, - ek — s=1 s s If ely > ek + Eke then xv is the optimal solution, STOP;

else return to step 1.

Optimality Cuts: From sections (1.5) and (1.6) we know that for any

xv E 7Z11 the dual to the second-stage function is

ns (x") = max{AT (lis — Ts)v) I A W < A


and the subgradient of Q3 (xv) is

V 5 = E 0Q, (e),

where is an optimal dual solution to the recourse problem. Since Q s is

convex, the subgradient inequality writes as

(23(x) (2(xv) 3 —

it is known by duality that

A .T9 - v ) = ( v

Combining the last two equations gives

(23( ) > h, — ) sTs Tsx.

Taking the expectation for ail realizations of random data gives

(23(x) ps AsT — s=1 s=1 s=1

where the first terni in the RHS represents e and the second term represents

Ex in Algorithm 2.1.

In other words. Q(x") uT (x — is a supporting hyperplane of Q at

:y'. This insight helps to build an increasingly better approximation of Q(x).

Note that the convex function Q(x) is equal to the pointwise maximum of ail

Opbmization ilfithods for Stochastic Programming 37

its finitely many supporting hyperplanes, and it is approximated by variable

O. which is bounded below by one of these supporting hyperplanes (optimal-

ity cuts). The idea here is to add the optimality cuts at each iteration to

construct a good description of Q at the optimality.

Feasibility Cuts: At a first stage solution xv, the feasibility set for the

second-stage problem is {y litry = h, — Tszy y > 0}. This shows that for

any feasible first stage solution x, there must be at least one y > 0 such

that Wy = h ., — Tsx. 1f this x is infeasible, the L-shaped method iteratively

generates separating hyperplanes, called feasibility cuts which only eut off

3: that leads to second-stage infeasibility. These hyperplanes are written in

terni of u (the dual multiplier of model (2.12)) and satisfy 0.T(/18 — Tsxv) > 0

and WTo. < O.

Sirice h, —Tsx" is assume(' to be infeasible, the optimal value to (2.12) is

strictly positive by duality, the optimal value of

max UT (h, — ne)

WT 0

s.t.

o- <

(2.14)

that is, o-T(//, —*ne), is strictly positive, which leads to inequality o-Ths >

°lyse. From the first constraint of (2.14), u satisfies 1,17Ta < 0, that is

aT W y < 0 for ail y > O. Therefore, u satisfies ail the requirements and cuts

off the x winch leads to second-stage infeasibility.


2.3.2 The Multi-cut L-shaped Method

In the previous section, we generated a single eut for the SP model. How-

ever, the structure of stochastic program allows the introduction of several

cuts instead of a single one. Here, we explain the multi-cut version of the

L-shaped method taken from [9].

Algorithm 2.2: Multi-cut L-Shaped Method

Step 0 Set r ks , r = 0, s = 1 . S.

Step 1 Set y = + 1, and for (xi', • ,O's ), solve the linear program:

min e

s.t Ax = h

Dex > de , L = 1, . , 7.,

0 > ees + ees x, es =1, 2 ks . s 1

x > 0, 0 E

(2.15)

Step 2 For s = 1 S solve the second-stage feasibility problem:

min w = eT v + eT y- .

s.t. Wy + y+ — = h 8 —

y,v+, - > O.

(2.16)

If w > O. let a be a dual multipliers, set r = + 1, = o-TT ,


dr = aThs , and return to stop 1;

else, w = 0 for ail s = 1, • • • , S, go to step 3.

Step 3 For s = 1, • • • S solve the second-stage problem:

(2,(e) = min

s.t.

(1, Y

Wy = h, —

Y > O.

(2.17)

Let the dual multiplier be A.

If 0: < NAT, (h, —ne) ; set k, = ks Eks = —PsXrsTs eks = P8A.Ths• If 0 > ps AT, (h, — ne) for ail s, xv is the optimal solution, STOP;

else return to step 1.

2.4 Sampled Based L-Shaped Methods

Stochastic Decomposition and importance Sampling generalize the L-Shaped

method for two-stage stochastic linear programming. In these methods, only

a subset of realizations are samples in each iteration to solve the subproblem.

This is useful for problems with a very large nmnber of scenarios or when the

random variables are distributed continuously. We give more details about

these methods in this section.

2.4.1 The Stochastic Decomposition Algorithm

In section (2.3) we introduced the L-shaped method as a stochastic decom-

position technique. The L-shaped method incorporates a given discrete dis-


tribution or an approximate discrete distribution of a continuous one in the

model. Then it generates feasibility and optimality cuts at each iteration to

obtain the optimal solution. Stochastic Decomposition (SD) algorithm by

Higle and Sen [41] employs the same framework as the L-shaped method ex-

cept that it uses a sample froni the distribution at each iteration to generate

approximate optimality cuts at the current solution.

Iteration k of the L-shaped algorithm solves a ma.ster problem of the form

unin c x + 0

s.t Ax = b,

0 > es + Esx, s = k — 1,

> 0, 0 E

(2.18)

where 0 is an auxiliary variable which takes the place of the expected value

function Q (feasibility cuts can be added to the master problem if it is re-

(luired).

To illustrate the SD method, let (xk , 0k) be an optimal solution to (2.18).

To construct an optirnality cut at Xk generate e and compute the cone-

sponding hk = h(e) and Tk = T(), and solve the following second stage

dual problem:

max{A(hk — Tkxk ) < g} . A

If we assume that the recourse structure is complete, then the dual feasible

set V = {A E < is nonempty and bounded, hence the optimal

value alliais to Ak (hk — Tkxk ) where Ak is one of the vertices of V. Let


vk = u {,\u.} be the set of dual vertices at iteration k, then for the

current solution Xk and ail previously generated samples s = 1 k-1,

solve

max{A(hs — xk) c

to °Wahl an optimal solution À. Then the new optimality cut at Xk would

be

AZ(hs —1'4= ekk Ex. (2.19)

The computation of cuts in the SD method is fast but inexact. In addi-

tion, the cuts may not be valid since they rua y not be lower bounds for the

expected value function Q. To fix this let L be a lower bound on the dual

optimal value with probability 1, then the old cuts 0 > esk _ i + =

1, • , k —1 are translated to

1 s = 1, • •• ,k —1. k k-1

Therefore the SD algorithm explained above would be:

Algorithm 2.3 Stochastic Decomposition

Step 0 Let k =1, Xk = O. V = O.

Step 1 Draw a random sample (just a single draw) ,e, k and solve

Ak = max{Ak(h(e) — Tk (es- )xk )111/T <

Set V = V U

For s = 1, 2, • • • , k — 1, solve = maxAcvlAk(h() —


Step 2 Create a eut as 0 > Esk=1 Ais,(t() —

Call the eut ek Ek x.

Step 3 Solve the Master Problem

(xk , = min cTx +0 s.t. 0 > ek + Ekx. Vk=1. 2.... K. xEx

Go to 1.

Since it is not possible to sample ail the realizations in sueh an algorithm,

the optimality cuts generated are only statistically valid. and convergence can

only be proven in a statistical sense [41].

2.4.2 The Importance Sampling

The Importance Sampling [17, 42] combines Monte Carlo sampling [68] with

an L-Shaped method. Importance sampling reduces the variance of the so-

lution estimate. In each iteration, a subset of realizations is used to generate

the feasibility or optimality cuts. Optimality cuts are only statistically

Probabilistic lower and upper bounds are eomputed and the algorithm ter-

minates when the bounds are statistically indistinguishable. Because ail the

scenarios are not tested, the second-stage feasibility is flot always guaran-

teed. The L-shaped method with importance sampling can be parallelized.

Computational experiments [42] show that importance sampling is capable

of solving problems with a large number of scenarios to high accuracy within

95% confidence interval of the solution.


2.5 Multi-stage Stochastic Programming

Many real-world problems sua as transportation problems and financial

planing cï.in be fornmlated as multi-stage or two-stage stochastic linear pro-

grains with recourse. In these types of problems, a stage usually represents

a tiine period. At the beginning of cadi stage, we know deternnnistically

ail the data in the current stage, but ail the data in the future stages are

stochastic. At the beginning of the first stage, decisions must be made before

the realization of the random data in the future stages. Once the random

data in a stage become known, correction (recourse) actions are allowed to

compensate the decisions for this stage made earlier. The effectiveness of the

clecision-making is measured by the total cost consisting of the deterministic

cost in the first stage and the total expectecl cost in the future stages.

In the previous sections we were mostly concerned with two-stage SP.

Here, we would like to extend the idea to multi-stage stochastic linear pro-

gram with fixed recourse. The general formulation of such a problem is,

min z = c1 x 1 + E [min c2 (W)x2 (w 1 ) E ii [min cH (w)xli '(wH-1 )] ...]

s. t. -1/171 :y 1 =hl

Ti xl w2x2 = h2 (2)

(2.20)

TH-1 (w ) xH-1 (wH-1) ± wHxli (wH-1) =hH (wH-1)

x1 > 0 , xv (wv) > y = 2 H.


where e1 is a known vector in Rn', h1 is known vector in R.Ini and ev(w) T = (c1) (w), le(w) 774 1 ) v = 2„ H, is a Ni, random vector and

each 11711 is a known x nv matrix. The decision x is dependant on the

history up to time v which is indicated by wv. As an example, we demon-

strate how to formulate a simple financial planning problem by multi-stage

stochastic programming.

A Financial Planning Example: Suppose we wish to provide for a child's

college education Y y-cars from now. We currently have 5b of budget to invest

in an y of I = 2 investment plans. After Y years we could have a wealth that

we would like to have exce.ed a tuition goal of $G. If it exceeds then it would

be equivalent to having an income of qa/0 of surplus money, while not meeting

the goal would lead to borrowing for a cost of r% of amount that is short.

We also suppose that we eau change the investments every v years so we

have H = Y/v investment period.

Suppose that we have 3 stages and 2 types of investments

S = s2, 531 s1 = 1, 2; s2 = 1,2; s3 = 1,2.

The major uncertainty in this model is the return of each investment i within

the period t. We denote the random variables to be t. w) and the decision

variable to be z w). The scenarios tree then would be represented as

follows:

The decision variables at euh node of stages 1 to 3 are represented as

t= 1 decision (i, 1) ; j = 1,2

t = 2 decision (i, 2, s i ) = 1, 2; = 1, 2


S3=2

Figure 2.1: Seenario Tree.


t = 3 decision 1, si, s2); j = 1.2,8 1 ,8 2 = 1,2.

Similarly we denote the returns as

t = 1 decision 1) ; j = 1.2

t = 2 decision 2, 81) ; = 1,2; si = 1.2

t = 3 decision 1, , s2) = 1,2, s i s2 = 1,2.

In general for H stages and N types of investment, the multi-stage stochas-

tic program consists of:

• The objective function for s i , sH is

max (s1, ((.1Y(si. — rw(s i . ... , sH)) SJj si

• The first-period constraint is the budget constraint. It indicates that

in the first period we can only invest as much as the initial wealth.

Id x(i,1)= b. iEN

• Constraints for periods t =- 1 H —1. for each s st_ i are

— 1, s, — 1,s 1 , 9e-2) = E :E(i, t — 1,81, ,St_). iEN

• Constraints for period H to meet the tuition goal of G' is

t-1, sH)x(1,t-1, s SH-i) — Y(Si, s 11)+71(s:h 8H) = C.

iEN

iEN

Optimization illethods for Stochastic Programming 47

2.5.1 The Nested Decomposition Method

The nested decomposition [76] extends the L-shaped method to solve multi-

stage linear prograumfing problems, where the uncertain parameters have

finite realization. Nested decomposition method solves the first stage and

treats the remaining stages as another multi-stage stochastic programming

and solve the problem recursively. At cadi iteration, the algorithm passes

forward the current solution to the later stages (forward passes), while infor-

mation passes backward in the form of the feasibility and optimality cuts to

ancestor problem (backward passes). There are different sequence procedures

that tell in winch order the problems corresponding to different nodes in the

scenario tree are solved. One of the most common sequencing procedure is

called Fast, Forward-Fast Backward. It vas initially proposed in [76] for de-

terministic problems and implemented for SP in [29]. This method suggests

that the algorithm should proceed in a direction as far along as possible until

an infeasible subproblem is fourni (when going forward) or when there are

no more cuts to pass back (when going backward). The rate of information

exchange is minimized in this method.

As with the L-shaped method, the optimality cuts need to be developed

for nested decomposition schema. Assuming the problem lias (relatively)

complete recourse the optimality eut, 07 1 > et!-1 — can be con- - -3 3 '

Optimization Methods for Stochastie Programming 48

structed with the component e and E as follows

( I)v-i)[Al lek ± kEDi(i) 3

Sk

i= 1

.ic'„v

" kkt

rk

i= 1

Ev-1 = 3 ( v-1)

Pvk \vrrv-1

kED,(j) Pj (2.21)

where Dv(j) denotes the stage y deseendent's of a seenario j at stage /.) — 1;

p, and u are the dual multipliers corresponding to the fixed constraints,

the feasibility cuits ; and the optimality euts in the multi-stage stoehastic

programming formulation. We assume that ail of the above variables have

finite upper bounds.

The nested decomposition method ean be implemented via parallel coin-

tutting. Parallel programming allows problems to be solved more quickly or

with more details. By parallelization, sets of scenarios are assigned to proces-

sors to ereate feasibility or optimality cuts simultaneously. Ruszczynski [65]

suggested a different way to parallelize the nested &composition algorithm:

all subproblems are solved at the same time to generate new cuts and current

solutions. This new information is then passed to a special buffer so that the

task can be perform asynehronously. This method significantly reduees the

idle time of the processors waiting for new information to corne in. Sec also

[7] for other parallelization sehema.

Chapter 3

A Decomposition Based on an

Interior Point Algorithm

3.1 Introduction

In the last chapter we went through simplex-based &composition algorithms

which are used in SP problems. However, the use of an interior point method

is often a. better choice in solving large-scale linear programming problems.

The linear SP problems are very large and normally have special structures

which are exploitable by interior point decomposition methods. This chapter

foeuses on an elegant interior point based &composition method, namely, the

Analytic Center Cutting Plane Method (ACCPM) and its implementation

on two-stage and multi-stage stochastie programs.

The ACCPM uses an interior point algorithm combined with classical

cutting plane methods (or column generation). The &composition and par-

titioning principles have been built in ACCPM takes advantage of the special

A Decomposition Based on an Interior Point Algorithm 50

structure of stochastic programming problems. The significant contributions

to this method are by Coffin and Vial [34, 33].

3.2 The Analytic Center Cutting Plane Method

The main contribution of the ACCPM is attributed to the minimization of

nondifferentiable convex functions approximated by the intersection of sup-

porting hyperplanes to its epigraph. The algorithm \\Thiel' is implemented in

ACCPM integrates two powerful techniques, namely, cutting planes and in-

terior point. Ï.1.1gorithm. The general mathematical format of an optimization

problem in which the ACCPM is applied to, is represented by

minff(x) = coT + fi (x) x E X = xo n, Xe}, (3.1)

where sets X, c R. L = ....r and functions --> 7, j = 1

are convex. Moreover we assume that X0 is a bounded polyhedron defined

by lineïn• inequalities Ax < b. Due to the convexity of the problem, the

epigraph of both f (z), i = 1, , / and X, , e = 0, ,r eau be approximated

by the intersection of their supporting hyperplanes. This gives piecewise

linear approximations to a feasible region as well as an objective function.

In each step, the ACCPM tries to refine these approximations. To achieve

this, in ea.ch query point (or test point), say ACCPM calls an oracle to

obtain the objective function value f(:), and subdifferential f Based

on the information which is provided by an oracle, ACCPM generates cuts

or improves the approximation to the objective function (from now on, we


assume that X = X0 nre=i Xt, is a bounded polyhedron).

3.2.1 The Oracle

As mentioned, the ACCPM uses a cutting plane technique to localize the

polyhedron within the current relaxed feasible region in which the minimizer

x* of (3.1) exists. Such polyhedrons are called localization„sets. Localization

set are defined by an outer approximation of the epigraph of the current LP

relaxation and bounded above by the best objective function value observed

so far. Computing the analytic conter of the localization set. at each iteration

provides the query point for the next iteration. However, the query point

might not be strictly feasible for the next iteration due to the introduction

of a cutting plane. In this case we have to find a new query point by adding

a feasibility eut. The information that is stored in the oracle at each query

point, in addition to the fact that oui problem is convex, helps us to construct

the current localization set as well as generating the feasibility cuts.

Given a query point î. the oracle first tests whether G X. If J; E X, then

the oracle computes fi (5;) and a subgradient E f GO with the property

(x) fi () + -yi (z — Va; E X. j = 1 (3.2)

The epigraph of f is defined by the intersection of the above inequalities.

Each of the inequalities provides an optimality eut. When the query point is

feasible, the objective function value at would be an upper bound on the

optimal value, i.e.,

fi (j;) i=


On the other hand, if the query point is not feasible, x X then the oracle

generates a hyperplane (^,(0 , -y) E 7? x Riz that separates § from X i.e.,

- 4 + 7o <O Vs E X. (3.3)

This Iwperplane represents a feasibility eut.

To have a better understanding of how the fea.sibility and the optimality

cuts improve the description of the feasible set and the objective fimetion let

us divide the query points into feasible and infeasible ones. Suppose that the

oracle lias been called K times and the following two sequences are made,

where K=IK U JR -,

/1( = { k Xk is infeasible} , (3.4)

{ k I Xk is feasible} . (3.5)

The optimality cuts at sk , k E 1K define a piecewise linear approximation

to the convex function f whereas the set of feasibility cuts at , k E /K

define an outer approximation of the feasible set X. These new definitions

of the objective function and of the feasible set provide a way to define a

relaxation to the master problem (3.1). We shah l call it the relaxed inaster

problem.

3.2.2 The Localization Set

Suppose that JK and define the .upper bound 6,„ = mink f (51 k E JR}

The following polyhedral approximation which is called a localization set at

A Decomposition Based on an Interior Point Algorithra 53

the iterate k is denoted by £K:

Vk E JK , i=1 ,

0 > - Xk ) 74e , (3.6)

éK > + • • + Z1,

The first inequality is a set of optimalitv constraints. The second set gives

the fea.sibility cuts. The third constraint makes z = z 1 + • • + zj fall bellow

the upper bound K• The last set represents the fixed constraints, winch

gives a boundary to the feasible region. These sets of constraints provide the

best outer approximation of the optimal set in (3.1).

Lower bound: A lower bound on the objective function cari be obtained

by using the dual of (3.6). Let us associate the dual variables aik > 0 to the

set of optimality cuts, ftk > 0 to the set of feasibility cuts, and 6 > 0 to the

fixed constraints in (3.6). The variable vectors ak. •k, 6 are dual feasible if

they satisfy the following system of equations:

E aik-Yi + kEJk

+ ATS =

E aik kEJ,„

= 1, i=1 J (3.7)

Multiplying each of the optimality, feasibility, and fixed constraints in (3.6)

1)3,7 aik , [I, 6, respectively, and simplifying the equation by the result in (3.7)

A Decomposition Based on an Interior Point Algoritirrn, 54

provides

_ i=i i=l kEJK

ik (fi (xk) _ „yika;k) ilk ey:)c ,ykxk) (3.8) ke it-

The right-hand side of (3.8) is a lower bound on the objective function (this

lower bound will automatically fans bellow OK in each iteration).

The initial convex feasible dornain shrinks repeatedly by the removal of

regions which caillot contain the optimmn. The ACCPM selects the query

point by fincling the analytic center of the localization set. There are also

other cutting plane methods such as Levin's center of gravity and the max-

imum inseribed ellipsoid, which maintain the volume reduction property as

well as the centering. The idea of cutting plane method originated with Kelly

[43].

3.2.3 The Analytic Center

The ACCPM finds an miter approximation of the polytope of the feasible

region at each iteration, the analytical center of this polytope (localization

set) is chosen as a query point. Cuts are generated from tins analytic center

of the set of localization and are added to the &finition of the polytope.

To compute the analytic center from now on, ive use the compact def-

inition of the localization set. Here the index K is dropped for simplicity,

therefore the localization set ,C is defined as

ty E TC1 I ATy > 01,


where A E TL' and c Given a point in L, define a metric d by:

1. d(y,£) = 0 if y is on the boundary of G,

2. d(y, G) > 0 if y is in intG,

3. 1f < c and G = fy e Rni I c' Any > 01, then d(y, G') < d(y,

This metric is dependent on the analytic representation of L. one choice

of this function would be

(1 (y , £ 71

aT : y)•y E G,

where a . is the j'il column of A. The logarithm of d(y,£) is called the dual

potential function:

(I) D (y, G) = log d(y. G) = E log (c:i — (!Ty)

(3.9) j=1

where si = ci — ary. The pair of interior points y' and sa = c — ATya in G

that maximize the potential function is called the anal tic center of G, i.e.,

op(ya. ) = max log d(y. G), yEC

or simply.

max{cbD (s)ls = c — ATy > 0}, (3.10)

where ØD(S) = E3n. log si . Note that (ya. sa) is a unique maximizer, since

the potential function is strictly concave.

A Decomposition Based on an Interior Point Algorithin 56

Let xa = (Sa) -l e, the analytic center ( lia , 3) together with xa satisfy the

following optimality conditions:

X8 e ,

Ax = 0,

AT y + s (3.11)

where X = diag(x). Therefore, an analytic center is the point that maximizes

the product of the distances to ail faces of the polytope defined by L. An

approximate analytic center relaxes the non-linear constraint Xs = e in the

first order optimality condition (3.11), i.e.,

X s

ATy s s > 0, (3.12)

where any pair of (x, s) defines an 17-analytic center.

Example: Consider the system G = fx I Ax = 0, eTx = n,x> which is

called Karmarkar's canonieal set. If x = e is in G, then e is the analytic center

of G. Tt would also be the intersection of the simplex {x I eT x = n, x >

and the hyperplane fx, I Ax = 01.

The analytic center can be alternatively computed as the optimal solution

of

max-Pp(x) j Ax = 01, (3.13)

A Deeornposition Based on an Interior Point Algorithin 57

where Op(x) = --erx Ein=i log xi . We &note the OD(S) in (3.9) is a dual

potential function whereas (;) p (X) is a primai potential function. The primai-

dual potential is then defined as

OPD = OP(-1.) OMS).

3.2.4 The ACCPM and Interior Point Algorithms

The ACCPM uses an interior point algorithm to compute the analytic center

at cadi iteration. The Primai Algorithrn, Dual A lyorithm, and Primal-Dual

Algorithrn are considered the most efficient interior point algorithms in prac-

tice. We avoid the detailed discussion about these algorithms and refer the

reader to the existing literature; for example see [77, 78].

A primai algorithm needs only a. primai initial point. This algorithm up-

dates the solution so that it converges to the analytic center, dual solutions

are by-products of the procedure. A primai algorithm is very efficient, spe-

cially when additional cuts are added to the definition of the localization set

and the ACCPM needs recentering.

A dual algorithm needs a dual-feasible initial point. The algorithm then

updates the dual solutions and the primai solutions corne as a by-product

of the procedure. Each solution is generated by solving a system of linear

Expiations. Hence, a sequence of simpler problems solves the original linear

program to optimality.

The primal-dual algorithm is very popular in practice. The algorithm

treats both primat and dual variables equally, and updates the primai and

the dual solutions in each iteration.


3.2.5 Multiple Cuts

The efficiency of the ACCPM improves if we are able to (i) add stronger

cuts in each iteration ; (ii) add multiple cuts simultaneously. The addition of

multiple cuts means allowing the oracle to return a set of cuts for eaeh query

point instead of just one. The ouality of the added cuts is measured by the

portion of the localization set which it eliminates. Suppose ; in iteration k.

Bk,yk - rk,

is placed. If -y > 0, the eut is deep; if -y < 0 the cut, is shallow; finally if -y ---- 0

the eut passes through yk and it is called a central eut. Deep cuts are the

strongest cuts among them.

Goffin and Vial [361 investigated the multiple eut generation in analytic

center cutting plane scheme. They proposed a method for finding a new

analytie center after adding multiple central cuts that required a solution to

a nonlinear program.

3.2.6 Adding and Dropping Cuts

Feasibility and optimality cuts in algorithms like L-Shaped method (section

2.2) eliminate either a portion of the feasible region or a contour of the

objective function in which the optimal solution does not happen to be. In

the ACCPM, cuts function similarly except that each time a eut is added ;

the coordinates of the analytic center are changed and hence a recentering is

required.

On the other hand ; the ACCPM includes a mechanism to prune the

A Deeomposition Based 071 an Interior Point Algorithm 59

redundant cuts where the problem is getting very large as a result of the

addition of single or multiple cuts in each iteration.

In this section we expiait' how to restore the anal tic center after adding

or dropping a cut.

Adding Cuts: Unlike many decomposition methods, in the ACCPM [33]

the relaxed master problems are not solved to optimality. Instead it seeks

the analytic center of the localization sets as a starting point for the next

iteration. This analytic center then, will be fed to the oracle to generate new

(nits. Generating cuts at each iteration of ACCPM means adding colunms

to the coefficient matrix of the relaxed master probleni. This changes the

definition of the localization set, hence the coordinates of the analytic cen-

ter. Therefore, designing an efficient way to find the new coordinates of the

analytic center alter adding cuts significantly improves the method. This

requires obtaining a new search direction after adding cuts.

To compute the search direction, let us partition the variables into two

sets. The first set of variables, winch corresponds to the old problem is de-

noted by z. s, whereas the variables corresponding to the new problem (which

differs from the old problem by a set of new cuts or columns) are denoted by ,new ,new . .

The first order optimality conditions to compute the analytic center as

A Decomposition Based on an Interior Point Algorithnt 60

we have seen in (3.12) is

Xs

As = 0,

AT y + s = e. (3.14)

Assume that a new cut BTy < r is added to the above problem. This causes

the old analytic center (y, s) not to be feasible to the cuiront localization set.

In fact adding a eut elhninates a portion of the dual space, which causes the

infeasibility of the current analytic center. This corresponds to introducing

a new variable in the primai space. It is often not practical to restore the

dual feasibility and maintain the centrality condition satisfied with the last

barrie'. parameter. However, the primai feasibility is always casier to restore.

The idea here is to restore the primai feasibility by adopting a ne -w search

direction Ar" and step size /3 that ensures s+,6z.Vx1ew > 0 gets to the interior

of the primai feasible region.

The analytic center after adding the cut BTy < to (3.14), is computed

from the following system of expiations:

ATy + s = e, s > 0,

BTy + = r, > 0,

As + Ba

( IaX s) (3.15)

where a and a are new primai and dual slack variables respectively, r =

A Deeomposition Basal on an Interior Point Algorithm 61

diag(a). Therefore, the coordinates of the new analytic center in the primai

space, say satisfies 1new = + Ax + Ba = O. x > O. c > O. We

adopt the direction Axne" winch is proposed in [56].

Arr e_ix2 AT (Ae_1x2 AT ) -1 Be,

=

where e = XS- I is a primai-dual scaling matrix. Hence,

AAr _Ae-1x2AT pe-1x2AT) -1 Be

AT' ± Ba =0, Ax = O.

Vector x is a strictly positive solution to the old problem, the maximum value

of /3 > 0 such that x"ew= x + /31rr > 0 is given by the ratio test

xi = min Anew

x3 Arxir < 0, .

Choosing the step size fl e (0,13max ) directs xnew to strictly primai feasible

interior, where the search for a new analytie center begins by using the primai

potential reduction interior point algorithm [78].

Coffin and Vial [36] propose a rigorous way to recover the primai fea-

sibility and the recentering alter adding a eut. The method is particularly

interesting when it brings the newly added variable to the primai space, say

into feasibility. It proves that a is a maximizer of a quadratic function

with self-concordant barrier[24], F(a). Hence, a feasible a cari be obtained

in finite Newton steps.


Dropping Cuts: The number of constrains in the relaxed master problem

increases as the cuts are added. This number increases by one (for a single

eut), or more (for multiple cuts) at each iteration. This makes the coin-

putation of the query point at each iteration a tedious ta.sk . The ACCPM

includes a mechanism for dropping a set of redundant cuts as the algorithm

progresses. Dropping the redondant constraints does not change the defini-

tion of £K.

The simplest way to check whether a linear inequality aT <b is redun-

dïmt is to solve the linear program

MaX

s.t • 1) <- j =1 rn, .1 —

Then <b is redondant if the optimal value is Inger than or equal to

b. Checking the redundancy of each constraint by solving a linear program

is very costly and normally a,voided. Another method defines an ellipsoid, F,

which is inscribed in the current localization set. If the maximum value of

o,Tx over F is smaller than or equal to b , the constraint ara; <b is redondant.

ln other approaches, constraints are dropped even if they are not known to

be redondant. Heuristics are used to rank the constraints in relevance, the

least relevant ones being dropped first.

In some cases, the ACCPM uses the inscribed ellipsoid method [3, 36] to

detect the redondant constraints. After detecting and pruning the redondant

constraints, the ACCPM lias to compute the next analytic center and recover

the feasibility. Here we show how to recover the dual feasibility after dropping

a number of constraints. Suppose that the first order optimality conditions


to find the current analytic center (1:, are given as

c, > 0.

± d = r, > 0,

AT + B 0 ; x > 0, à > 0,

= rd = p.. (3.16)

Alter dropping the cut Bny < r ; we need to compute the new analytic center

that satisfies

Xs = 7/:

ATy + s = c ; > 0,

Ax = 0, x > 0. (3.17)

Since erj .) are dual feasible (dropping cuts (1oes not change the definition

of the localization set), we use them as the initial point to the dual Newton

method to search for a new analytic center. Let the new iterate be x =

y =V + dy, and s = .5-;" + ds , thon (3.16) reduces to

Adx = —Bd.

Sdx + Xds =

ATdy ds = 0,

A Decomposition Based on an Interior Point Algorithra 64

where dx.ds is ignored. The above system has a solution

dy

ds = —ATdy,

da; Xds+

where V = AXÉ-1 AT. To compute the step size in each of the directions

(1s, dx, let us denote

fi = + X -1 S-1 1),

f2 =

Therefore dx = Xfi and ds = Sf2 . Taking the step size of là along dx

and Ilft2ii along dy and ds, where t E (0, 1) gives us the new iterate:

t , t X 1; + ,

l a x = X ( C ±

Il fi II fi) Il i 1 l s , ± t. d s = S(e +

11:ti.2 11 fi) I I J2 11

t t

ij + dy = 9 ATds II f2 II II fi II

winch satisfies ATy + s = e, x > 0, s > 0, hence y is dual feasible. Primai

feasibility is not achieved since Ax 0.


3.3 The ACCPM for Stochastic Programming

with Recourse

There are different formulations and corresponding solution methods for

stochastic progra.mming. One formulation is based on tue deterministic

equivalent. Deterministic equivalent is a large-scale linear program winch

can be solved b simplex or interior point methods. A deterministic equiv-

aient problem includes ail constraints and scenarios, hence it becomes very

large, specially in the case of multi-stage stochastic progriumning.

On the other hand, stochastic programming with recourse lias I: special

structure that can be exploited by decomposition. The primal of a recourse

problem lias a dual block angular structure, where its dual has a primai block

angular structure. These structural advantages makes the dual problem to

be handled by the Dantzig-Wolfe decomposition method. Equivalently, the

primai can be tackled by Benders decomposition or Kelly's cutting plane

method. Other algoritlmis such as the L-shaped method are designed to

exploit this special structure.

The ACCPM combines the interior point algorithm with a cutting plane

method, providing a fast algorithm for solving two-stage stochastic program-

ming with recourse [3]. The decomposition and partitioning principles have

been built in the ACCPM and it takes advantage of the special structure. In

ACCPM, new cutting planes are generated from the analytic center of the

set of localization as compared to Kelly's cutting plane method, where cuts

are generated from the optimum of a linear programming relaxation. The

analytic center contains the information from the past iterations and it is

A Decomposition Based on an Interior Point Algoritirm 66

a guide to the convergence of the method to optimality. In the next sec-

tion we investigate the use of the ACCPM for solving two-stage stochastic

programming with recourse.

3.3.1 Stochastic Programming with Recourse

The general formulation for two-stage stochastic programming with recourse

is

min z = co Xo Q (X0, ) d:1:0eeE (3.18)

SA. A0:1:0 > bo ,

where :ro is a first stage decision and is a random variable with finite sup-

port E. Furthermore, we assume a feasible recourse, i.e., for some x 0 , with

Aoxo feE ., Q (4, 4) chro is finite.

Moreover, the function Q : 7Z" x E R, is defined by

Q (x) min {J(03; I BWx > 140 — A()3;0 1 . (3.19)

Since E is a finite problem, (3.18) can be written hased on its deterministic

equivalent, i.e.,

min z = cPco +

s.t. il0X0 >

i=1 (3.20)


with

L (x0 ) = min { eiTx Bi x > h — Aixo } j = 1 1 (3.21)

where x is a second-stage variable.

The relaxed master program after k iteration, as it is described in section

3.1, is defined as

min z = c0 X0 E zi i=1

s.t. A0 X0 > bo , zi > (x0 _ xok) k E jKi

-yoki (-yik ) T (x0 — xt;) E i 1.

The dual formulation to (3.22) is

(3.22)

max bonu0 + (fik _ ( ,yik )Tx0k ) ,ik E

i=1

k■T k k ' ) X 0 !Li

kE Jk

_

i=1 kcJi, kElk

k k "Yi = co, (3.23)

E = 1, 1 = 1 /, kEe

uo , A125, > 0, i e

where A li' are dual variables associated with the optimality cuits, and 145 are

dual variables associated with the feasibility cuts. The query point x10̀ is a


solution to the relaxed linear program(3.22), at winch the oracles are called.

The answers of the oracles are then acide(' to (3.22) to get a better approxi-

mation.

is the lower bound to (3.22) which is given by any feasible solution to

(3.23)...f1( is the best feasible objective to (3.19) winch is so far known, i.e.,

1_ E J iy_.k 1 , i Aoxo > bo . 10; o I 7' 1' = min c0 3, 0 k=1,...,K

The optimum of (3.19)„ occurs at 71( < 7* <

3.3.2 The Potential Function and the Analytic Center

For the sake of simplicity, we drop the iteration index k and rewrite (3.22)

as the following linear problem. Further, we assume that the problem lias a

bounded feasible region:

min z = cTx

s.t. As — s

s > O.

(3.24)

where s is a primai shck vector anci TT — (x', if). The dual to

(3.24) is:

ax bTy

s.t. A .,_•T y = c,

y > O.

(3.25)

i=1

A Decomposition Based on an Interior Point Algorithro, 69

The set of localization is

£2 = I > cTx, > .

The potential function is defined by

.1)(x, = log ( c'Tx) log si . (3.26) j=1

The analytic center of the set of localization C is the unique solution to

vx0 = O. i.e.,

— cTx + ATS-1 6 = O.

Therefore we eau define the dual variable y as

y = — cTx)Se > O.

The analytic center eau equivalently be computed from the system of equa-

tions obtained from the first order optimality conditions

AX -s = b, AT y =

S y = (2 — er e, (3.27)

where S = diag(s). The above conditions express the primai feasibility, dual

feasibility, and the centering condition. To get the fuite convergence to the


analytic center, we need to relax the centering condition to

— (2— (.T.r) e M

where 0 < < 1. The ACCPM algorithm is then described as follows:

Algorithm 3.1: The ACCPM Algorithm

initialization

A feasible point ro and its corresponding upper and lower bounds f, z,

The initial set of localization Gi•

While z — 2> c

Compute the n approximate analytic center of L. say r, and its cor-

responding dual solution y.

Update the lower bound z by using z = max{17y,

Call the oracle at point r and update the upper bound 2 if there is

no feasibility eut added.

Update the localization set G 2 .

End-while

3.3.3 ACCPM on Two-Stage Stochastic Programming

The implementation of ACCPM on two-stage stochastic programming [3]

uses a variant of Karmarkar's [20] primal potential function on the dual of


(3.25). It computes the dual analytic centers by a primai projective algo-

rithm. The algorithm obtains an efficient search direction to recover feasi-

bility and find the new analytic center after adding new cuts. ft also has

the ability to remove the redundant information 1)y dropping cuts. The al-

gorithm identifies the redundant cuts by the procedure proposed on [351. ft

then efficiently restores the feasibility and find the new analytic center for

the current localization set.

The algorithm lias a polynomial thne complexity and it is faster (in the

number of iterations) than the class of decomposition methods, namely, Ben-

ders' decomposition and Dantzig-Wolfe decomposition, winch are used in the

stochastic programming context.

3.4 Two-level ACCPM for Three-Stage Stochas-

tic Programming

3.4.1 Column Generation on the Dual

R,ecall the multi-stage stochastic programming formulation in (2.20). To

formalize the two-level ACCPM, consider the following three-stage SP:

T .(1) (1) (1) (2) (2) (2) min cx 2—k=1 ck Pk Yk Lik=lek Pk Yk

s.t. Ar =

+ 1171 y(1) =

112Y(1) + 14/2 Y (2) = 112, > on

A Decomposition Based on an Interior Point Alyorithm 72

where the stage 0 is deterministic, the number of scenarios in the first-stage

ism, and the number of seenarios in the second-stage is n = rn2 .X s the

variable vector associated with stage 0, whereas y (1) , y (2) are variable vectors

associated with the first-stage and the second-stage, respectively. Denote the (2) (2)1 th en q( 1 ) by [c (11) . , c( ,, ) lig)] and (2) by [c(12) P.12) • Pri tnen

the three-stage stochastic program cari be reformulated as

min

s. t.

(3.28)

(3.29)

(3.30)

(3.31)

(3.32)

The dual to the above program is

max br A + hir + h.T p, (3.33)

s.t. AT À + TrF < c, (3.34)

+ "'Lu < q, (3.35)

14'11/ < (12, (3.36)

where À is the dual variable associated with (3.29), Ir is the dual variable

associated with (3.30), and p, is the dual variable associated with equality

(3.31).

The stochastic linear recourse program in (3.28)-(3.32) is a determinis-

tic equivalent program and its constraint matrix lias a dual block-angular


structure, moreover its dual constraints matrix in (3.34)-(3.36) has a primai

block-angular structure. Therefore, the stochastic linear program in (3.28)-

(3.32) is amendable to an approach by Benders' decomposition whereas the

dual program in (3.33)-(3.36) can be tackled by a Dantzig-Wolfe reformula-

tion and column generation technique.

Since the structure of the dual program is more supportive of decom-

posing the SP along the scenarios, we start the decomposition using the

dual. Hence the neural decomposition choke to tackle the problem would

be Dantzig-Wolfe. The combination of Dantzig-Wolfe reformulation, AC-

CPM and colunm generation would solve the three-stage problem in two

levels. In the first level we decompose the problern into a master program

(sec section(2.2.2) for more details) which is solved by ACCPM (section 3.2)

and subproblems. Each subproblem thon again decomposes into a master

program and subproblems. Figure 3.1 illustrates the two-level &composition

along the scenarios for a three-stage SP. Figure 3.1.a pictures the first-level

partitioning whereas the Figure 3.1.b shows how the subproblem in the sec-

ond stage is decomposed into a master program and subproblems.

To observe how the decomposition is donc, in the dual prograni, let the

first constraint, AT < c, to be kept in the n'aster (as the complicating

constraint). The master program is obtain by a Dantzig-Wolfe reformulation

on the objective and the complicating constraints in each stage (sec section

a) b)

Master Problem

Subprobleml Subproblet

I Suhp °bleui

Subproblerni Subproblenl Subprobleml Subproblemi Subprohleml

A Decomposition Based on an interior Point Algorithia 74

Figure 3.1: Two-level decomposition along the scenarios for a three-stage SP.

2.2.2 for more details). Then the first-level subproblem appears as

min — Ii5JT ir + h p.

s.t. 117T + T jia <

11/11-1 < (12 (3. 37)

where :7; is the primai value from the stage 0 to be carried over to subproblem

from solving the master program in the first level. This subproblem can be

decomposed into independent subproblems for cadi first-stage scenario. To

achieve this, let the dual variable in the first-stage TC , to be decomposed

to m distinct vectors for cadi sce,nario; i.e., 7r (1) = [741) [7r 2) if (2)1

k2 1 , 7r (771) = [Tr Irn) 7r i(rm ) ] . Now corresponding to cadi vector

variable 7r (i) ; i = 1, , m, we make a dual vector in the second-stage, i.e., fi ()) bil (11) t12)] ft(2) {142) , , p4n2)] fi (m ) [itr) J. Each

(k1 ) 7, 7 ( 2 )

vector = 1 m , is made by concatenating ail the m vectors in

the second level which is associated to one parent node in the first, level.


Therefore, by concatenation the pi , j = 1, . , m2 , vectors in the second level

couvert to ft(i) i = 1, , m, vectors each associated to a parent node in the

first level. As a result the subproblem in (3.37) becomes

min >I Rhi — +

s. t. 1V7T(i) + T ft(i) «1 (1i ) , = 1, m

tiirift ( z ) < (h(22) Tri. (3.38)

Since the objective function and the constraints in (3.38) are separable in

terms of the variables the resulting subproblem reduces to Tfl independent

subproblems:

min (h 1 —171 ,Ï;)T'F (i) +

s.t. wb(i)-+Trift(i)<e),

1171iii (i) < (gi) . (3.39)

Such disaggregation adds m convexity rows to its master program. In

the next level, each of the m subproblems are decomposable in tenus of

the second-stage dual variable p. This tinte, in each subproblem we keep

the constraint + < e) , in the n'aster program level as a

complicating constraint. Each master progra:m in the second levet contants in

convexity rows as a result of the disaggregation in the second level. To have

a better understanding of how the second-stage subproblems are created,

we look at a specific subproblem in the first-stage, the ith subproblem for

example as it appeared in (3.39). For titis problem, let u to be the dual


multiplier associated with constraints 1,1Dr(i) + Trie <q then the ith

subproblem eau be reformulated as

min (h2 — T2a)Tp,U )

s.t. Wrip,U) < (.0 j = 1 ?TL

Similarly, the above program decomposes into m independent subproblems

min (h2 —

s.t. 111.;P(3)

for j = 1 m. The above decomposition is repeated for each of the rn

subproblems in the first-stage, as a result there are m 2 subproblems in the

second-stage.

The n'aster programs in the first, and the second stages are solved by the

ACCPM, where it passes the query points (sec section 3.2), as central dual

prices to subproblems. In return, the subproblems in each level generate

columns to be added to the master program.

3.4.2 Benders' decomposition of the Primal

The Benders' decomposition principle is based on a variable partitioning

procedure in a large-scale primai problem, sec [4]. The method partitions

the variables by fixing the so-called complicating variables. This reduces the


given problem to a more tractable, and hence casier, problem to solve. The

latter problem is parameterized in ternis of the vector of the complicating

variables. The dual to this problem forms the subproblem. In each iteration,

the subproblem passes forward the current dual solution (for a fixed vector

of complicating variables) to a master program in the form of a Benders eut

that utilizes the linear programming duality to reflue the value of the vector

of complicating variables.

In the particular case of MSP problems, the paditioning would be in

terrils of the variables in cadi stage. Shnilar to what we implemented on

the dual problem when we used the column generation technique, we the

decompose three-stage stochastic problem:

max cT X

S.t . A x Tl x

B

X. , y2 > 0.

+q2y2

= b

+ 1171 Y1

+Tel +1472 ://2 = h2,

(3.40)

in two levels by using Benders' decomposition. This can be donc by first con-

sidering the vector of the first-stage variables — namely yi = '/A }

as the complicating or fixed variables. This forms the following subproblem,

Tjx

B

X. 712 > O.

(3.41 )

+(hTh -1- (12y2

+14/2 Y2

= min ( T J . - .9PB1ffl i )

S.t. A x

A Decomposition Based on an 1nterior Point Algorithm 78

given that Yi is fixed at

Sol-ving (3.41) provides au upper bound on the original problem (asssuming

that this subproblem is bounded). Therefore, the solution to the original

problem consists in solving the above subproblem for the Lest y]. value, and

it is given by the solution of the following (first) master problem:

[111PB] : min yl>0 :' Z PB 1 (Y1)'

Note that the dual feasibility of the subproblem is independent of the value

Let À. 71", and it be the dual vectors associated with the primai constraints

of stages zero, one, and two, respectively. `nen, the dual constraints of (3.41)

are:

AT +T"7 +BT,u. <C.

1-47Tp, <

The dual solution of (3.41) is given by an extreme point (4, -Fp , p E P,

for which the objective function takes value

bAp — W1 Yibtp (h2 TWitp qiTh.

A Decomposition Baser] on an Interior Point Algorithm 79

Therefore, the optimal solution to the subproblem cari be interpreted in ternis

of the best extreme point in P, that is:

'SPB1(Yi) = maxpE p

= min

s.t.

bAp + (h1 — 14711.ii)irp + (h2 — Tei)Pp+ giTh

el

ei > + (h1 - Wl yi)irp + (h2 - T2Y1)Pp + (eh,

p E P.

The first master problem, which provides a solution to the original problem,

(;an flow be written as follows in terms of the extreme points in P and the

vector of complicating variables 71 1 :

2;;L' 1 PB

s. t .

min el

el> + (hl - np + (h2 - T2yi)pp + q1y1, p E P

iii > O.

To complete the two-level Benders' decomposition, note that problem

(3.41) cari also be solved by Benders' decomposition b -y taking the vector of

the second-stage variables 712 = fy?, y y n” as the complicating variables.

Therefore, given that 712 is fixed at F2 , the second subproblem appears as:

-* -SPB11(i 172 ) = min +gill i +q2Y2

s.t.

= b

= h 1 - 14/1

= h2 — Teh — 1472F2,

(3.42)


Solving (3.42) provides an upper bound on problem in (3.41) (assuming that

the second subproblem is bounded). Therefore, the solution to the first

subproblem given 7j consists in solving the above second subproblem for the

besty2 value, and it is given by the solution of the following (second) master

pro bleui:

plIP/31(y1 )] : min Y2 >0

-* 'SPB11(-e 1 ,y2)•

The set of dual constraints for (3.42) appears as:

AT À +/T7r +Brp, <G.

The dual solution of (3.42) is given by an extreme point (4,7,7 ,1.1,J, q e Q,

for which the objective function takes value

bAq + — ) 1ra + (h2 — Tel — 1V2 Y2)tiq + (1.1 Ùi + Q2 F2.

Therefore, the optimal solution to the second subproblem cari be interpreted

in ternis of the best extreme point in Q. that is:

,* ‘'SPB11(-gi ,1/2)

maxgeQ bAq (h1 — W1Tir)irq (h2 712y 1 - w2 ://2)flq + (11 Th + Th

min (1) 2

e2? b + (h1 — 1)7r + (h2 — T2F 1 — 1472 Y2)I-tq +(11j 1 + (1.2 Th:

E Q.

The second master problem, winch provides a solution to problem (3.41),

A Decomposition Based on an Interior Point Algorithm, 81

can now be written as follows in ternis of the extreme points in Q and the

vector of complicating variables y2 :

,* -AIPB1(-y1) = min e2

s.t. 02> bAq + (hi — Ti1) 7rq + (h2 - - W2 y2)ilq

+ql ± q2 Y21 E Q

Y2 > O.

The two-level Benders' decomposition consists of two master problems

and a subproblem, where the master problems are solved by the ACCRM

and the subproblem is solved by the simplex method.

3.5 Conclusion

Marly real-word problems require that decisions be made periodically over a

finite time-horizon in the face of uncertainty. These types of problems are

°lien formulated as multi-stage stochastic programs (MSP). The determinis-

tic equivalent problems for 1MSP are very large, and direct solution methods

sud i as the simplex or interior point algorithms are very costly to be applied

to the problem. Given the size of the problem instances involved, decompo-

sition and partitioning methods are commonly used in MSP to reduce the

size of the problems a.t the time. The decomposition methods replace the

direct solutions of a large-scale problem with a: sequence of solution of smaller

problems.

In Chapter 2 and Chapter 3, we considered two forms of decomposition


approaches: the Benders and the Dantzig-Wolfe algorithms, winch are viewed

as simplex-based decomposition algorithms; and the ACCPM. In the remain-

ing Chapters we present the variant of the two-level &composition algorithm

to solve problems. This algorithm is implemented by using ACCPM,

Dantzig-Wolfe and Benders methods. Finally, the numerical results of these

decomposition techniques are compared.

Chapter 4

A Two-Level Interior-Point

Decomposition Algorithm for

Multi-Stage Stochastic

Capacity Planning and

Techn.ology Acquisition

4.1 Introduction

Manufacturing flexibility is recognized as one of the key strategies to address

uncertain future product demands. Flexibility is regarded as an adaptive

response to an 'incertain environment hi which it enhances the competitive

advantage of a manufacturer or a service provider. Therefore, there is a

growing need to investigate the strategic aspects of flexibility.

A Two-Level Interior-Point Decomposition Algorithrn for Multi-Stage Stochastic Capacity Planning and Technology Acquisition 84

A body of scholarly research lias developed over the years, sec for example

[14, 51, 66], to explore the many lacets of manufacturing flexibility. Among

them, [66] provides a taxonomy for the classification of different kinds of

fiexibilities throughout the entire manufacturing chain, from the acquisition

of the raw material to the delivery to the final customers. This chapter em-

phasizes the expansion, product, volume, and market flexibilities according

to their classification.

Expansion flexibility allows many companies to adopt a consistent growth

strategy. Such organizations retain flexibilitv by investing slowly and keeping

their options open for graduai growth. The aforementioned trend is seen

particularly in the electric utility industry where the graduai investrnent on

the capacity balances out the advantage of having a costly large plant. By

having a large plant instead of two or three smaller ones, companies reduce

the unit costs, and therefore, the coherent decision would be to gradually

invest in the new capacity and add a large plant cadi time.

Alternatively, product and volume flexibilities help compa nies to be re-

sponsive to market changes by quickly introducing a new product design or

adjusting the production volume upward or downward within a wide vari-

able range. Since the future design and volume of a product is unknown,

an acquisition of flexible machinery - or in other words, flexible technology

is essential to being more adaptive to change. The car industry is a good

example of a flexible technology acquisition that supports the assembly of

diffèrent models on the saine production hue.

Finally, market flexibility is regarded as overall manufacturer adaptabil-

ity to the environment that fluctuates constantly. The market fluctuates

A Two-Level Interior-Point Decomposition Algorithm for M'alti-Stage Stochas tic Capacity Planning and Technology Acquisition 85

because of the fast pace of technological innovations, shorter product life

cycles, and changes in consumers' behavior. As the market changes in a dy-

namic fashion, companies ma y neeci to adopt a new product design, acquire

a new technology, or cope with volume fluctuation.

To capture the different aspects of market tlexibility in tire face of this

dynamic demand, this chapter focuses on tire role of product, volume, and

expansion flexibility in the context of the multi-stage stochastic program. By

using tins multi-stage program, one cari capture the evolution of the demand

across the planning horizon. 'Technology ïtcquisition or facility expansion can

be obtained once in cadi period before the realization of the random demand,

tire recourse action would be to either have an inventory for excess production

or to be penalized for flot meeting the demand of certain customers.

Given a finite number of stages, scenarios and their probabilities, the

capacity expansion and technology acquisition problem is a large-scale opti-

mization mode', which is tractable by decomposition. Decomposition meth-

ods are ways of solving a sequence of optimization problems virose size vary

increasingly based on tire information feedback given by the method used.

Hence, the decomposition algorithms reduce the size of the problem to make

it solvable. The classical decomposition tools for linear programming prob-

lems trace back to Dantzig and Wolfe [54] and Benders [4]. These &compo-

sition principles are translated into the L-shaped method [72] in stochastic

linear programming. The L-shaped method iras been extended to the nested

&composition method and developed as a computer code for linear multi-

stage rec ourse problems [6, 29].

The L-shaped method is viewed as a simplex-based decomposition princi-

A Two-Level Interior-Point Decornposition Algorithm for Alulti-Stage Stochastic Capacity Planning and Technology Acquisition 86

pie in contrast to a method such as the analytic center cutting plane method

(ACCPM) [33], which is based on an interior point algorithm. The ACCPM

combines the interior point algoritlmi with the cutting plane technique to

provide a powerful tool for solving large-scale optimization problems. The

ACCPM was first implemented for solving two-stage stochastic programming

with recourse in [3]. This specialized implementation is successfully used for

practical applications suc:h as the telecommunication network problem in [1].

A two-level decomposition algorithm via ACCPM is also addressed in [?]

to solve production-distribution problem. In the prescrit chapter, we impie-

ment a new two-level decomposition method based on the ACCPM to solve

large-scale capacity planning and technology acquisition problems imder un-

certainty. The resulting algorithm is then compared against the two-level

shnplex-based classical colunin generation technique.

The decomposition methods that are used for solving stochastic programs

eau be classified into three categories: i) direct methods that solve the de-

terministic equivalent to a stochastic program by using a simplex or interior

point algorithm [8, 5, 15], ii) cutting plane algorithms that are mostly based

on the Benders' decomposition of the primai formulation [3, 72, 79], a.nd iii)

Lagrangian based, dual decomposition methods [63, 64]. Cutting plane al-

gorithms such as the L-shaped method are initially designed and utilized for

solving the two-stage stochastic program and cari extend to multi-stage as

the nested decomposition method [6, 29]. The nested decomposition method

is a simplex-based decomposition algorithm.

hi the present chapter, we consider a two-level interior-point-based de-

composition algorithm that consists of two master programs and a subprob-

A Two-Level friterior-Point Decornposition Algorithrn for Multi-Stage Stochastie Capacity Planning and Technology Acquisition 87

lem. Both master programs have Dantzig-Wolfe reformulations that are

solved by the ACCPM to provide central dual puces. Note that the combina-

don of the second n'aster program and its subproblem acts as a subproblem

for the first master. Alternatively, oui' problem is also decomposed by using

the classical Dantzig-Wolfe reformulation and column generation technique

where both master prograrns are solved by the simplex algorithm for the

sake of comparison. In both decomposition approaches, the subproblem in

the second le.vel is always solved by the simplex method. In the classical

Dantzig-Wolfe decomposition approach, colunms are generated by using the

dual extreme points of the restricted master program, whereas the two-level

ACCPM takes advantage of using the central dual prices, i.e., the dual cen-

tral points that lie in the relative interior of the dual feasible region. This

strategy introduces a different venue in approaching the column generation

scheme. Note that the recent developments in classical Dantzig-Wolfe de-

composition and column generation focus on regularization and stabilization

techniques [61] to provide better dual prices.

Furthermore, our implementation is a serious step towards the efficient use

of the interior point method within the framework of the multi-stage stochas-

tic program. Previous efforts [5, 151 are limited to utilizing the interior-point

method in solving the deterministic equivalent to a mufti-stage stochastic

program. However, our algorithm uses an interior-point method in a cut-

ting plane (column generation) context radier than a direct solution to the

deterministic equivalent problem.

The present chapter is organized as follows. In Section 2, we provide the

formulation of a multi-stage technology acquisition and capacity expansion

A Two-Level Interior-Point Decornposition Algorithin for Multi-Stage Stochastic Capacity Planning and Technology Acquisition 88

model. In Section 3, we address a two-level decomposition approach that

leads to the column generation technique. ln particular, we introduce the

notions of the restricted n'aster problem and the subproblem in cadi level. In

Section 4, we deal with solving the restricted master programs by using the

ACCPM, and present â pseudo-code for the two-level ACCPM. In Section

5, we discuss the computational experiments; and finally, in Section 6, we

present our concluding remarks.

4.2 The Technology Acquisition and Capac-

ity Expansion Mode!

The technology acquisition and capacity expansion model cari be viewed as

a single plant model with multiple production capacity and a capability for

inventory holding. The goal here is to maximize profits in mufti periods of

uncertain product demand. The multi-stage stochastic program consists of

T planning periods (stages) with stochastic demand and a stage zero that is

characterized by a deterministic demand. The indices, variables, and input

parameters used in the model are denoted as follows:

Indices: N: number of products. T: number of stages (periods) in the planning horizon. Kt: number of scenarios in planning stage t, t = 1, • ,T T. cif(kt ): inventory index for scenario kt.

A Two-Level Interior-Point Decomposition Algorithm for Multi-Stage Stochastic Capacity Planning and Technology Acquisition 89

Variables: Xi0:

X iikt:

Iitkt :

mnount of product j manufactured in stage 0, i = 1, • • • N. amount of product i manufactured in stage t under scenario kt , i = 1 ; • • • N; t =1 ; • • • ,T, kt = 1 ; • • - Kt . inventory of product i at the end of stage 0, j = 1, • - . N. inventory of product i at the end of stage t Linder scenario k t ,

= 1, • • • , N. t = 1, • • • , T, kt = 1, • • • , Kt . unsatisfied demand for product j in stage 0, i = 1, - • • , N.. unsatisfied demand for product j in stage t under scenario

= 1, • • • , N, t = 1,• • • ,T, 1,- • • Kt . additional capacity at the beginning of stage t for product

Yilkt :

Uu : t = 1. • • • , T.

Parameters: Uo:

Pukt:

hitkt

cdukt :

Ptkt:

initial capacity at stage O. marginal profit or price for product i at stage 0, i = 1, • • • , N. marginal profit or price for product i at stage t under scenario kt; j = 1, • • • , N. t = 1 ; • • • ,T, k t = 1, • • • Kt . unit holding cost for product i at stage 0, i = 1, • • • N.. unit holding cost for product i at stage t under scenario =1 ; • • • ; N. t =1,• • • ,Tkt =1.• • • ; Kt .

unit penalty cost for flot meeting the demand for product i at stage t under scenario k 1 . i =1 ; • • • ; N, t =1,• • • ; T. kt = 1 ; • • • , probability of scenario kt in stage t ; t = 1 ; • • ,T, kt 1 ; • • • ,K.

Given the above definitions, a general T-stage model eau be formulated

A Two-Level Interior-Point Decornposition Algorithm for Multi-Stage Stochas tic Capacity Planning and Technology Acquisition 90

as follows:

[TS111] : max E(pio X

h,i0 — Cdio Yio) —

N T

cit i=1 i=1 t=1

N T Kt N T Kt

Ptk .Pitk X ik — Ptk hitk I itk i=1 t=1 k=1 N T Kt

— E E E Ptk CditkYitk i=1 t=1 k=1

i=1 t=1 k=1

s. t. (4.1) i=1

Xj0 < = (4.2)

E Xtk < uo + N T

t = • ,T, kt = 1, i=1 t=1

+ Xi1k1 lilki <

iita(kt ) XitkÉ 1 itkt < ditkt ,

liTkT =

X.Y,I,U > 0,

(4.3)

i = 1 • • Y, k1 = 1. • •

(4.4)

i = 1,••• IV, t = 1, • • T.

a(kt) = 1, • • • Kt _ i , (4.5)

kT = 1, ••• KT , (4.6)

where = EiN__ 1 (dio — 40 ) and 1/itk, — Xitkt )• In = EiN-1 ET -1 Eikc-ti(ditkt

[TMS], constraint (4.1) refers to the capacity constraints in stage 0, while

constraints (4.2) specify that the demand must be satisfied as much as possi-

A Two-Level Interior-Point Decornposition Algorithm for Multi-Stage Stochastic Capacity Planning and Technology Acquisition 91

hie in stage O. In stages t = 1, T, constraints (4.3) represent the capacity

restriction. Constraints (4.4) force the demand to be satisfied in stage 1, and

constraint (4.5) forces the demand to be satisfiecl in the rest of the stages.

Constraints (4.6) insure the inventory to be zero at the end of the planning

horizon.

4.3 A Two-level Decomposition Algorithm

The solving of multi-stage stochastic programming problems requires efficient

algorithms, since the resulting MSP problems are extremely large scale pro-

grains. Software packages built on simplex or interior point algorithms eau

solve instances of stochastic programming problems efficiently if their special

structure is fully exploited. However, the exploitation of this structure is not

always evident in stochastic programming. The recourse structure in [TMS]

formulation enables programmers to take advantage of the dual block-angular

structure of the multi-stage stodiastic program to decompose it into two lev-

els. To demonstrate how the &composition is performed, without loss of

generality, we consider the following three-stage formulation of [TMS]. This

formulation consists of a deterininistic stage (stage 0) and two other stages

with stochastic variables, where the first stage has ni scenarios while the

second stage lias n = in 2 scenarios. In the three-stage inodel ; x is the vec-

tor of variables associated with stage 0, whereas e • • , yn1 ,1 and

Y2 = A, • • • , yri2 1 are the vectors of variables associated with the first,

stage and the second stage, respectively. The cost structure consists of the

direct cost e of stage 0, and the expected costs of the stochastic variables in

A Two-Level Interior-Point Decornposition Algorithrn for Mati-Stage Stochastic Capacity Planning and Technology Acquisition 92

the first stage and second stage denoted by (h and q9, respectively. Then,

the generic primai three-stage stochastic program can be formulate.d as:

[Primai] : max cT +(hYl. 1- (12

s.t . A a; = b

Tl x + W1 Y1 =h

nyi +14'2Y2 = h2

x, Y2 > O.

To case the notation, we renaine the right-hand sicle of the above model

as: b = = c„ and h2 = cp . Then, the generic dual problern is the

following:

[Dual] : min cT `-' ,

itt ,

7r "

si AT +TTR.

1,11Tir + Tr

where À. -A- and il are the dual vectors associated with stage zero, stage one,

and stage two, respectively.

The primal stochastic linear recourse program is a deterministic equiva-

lent program, and its constraint matrix lias a dual block-angular structure.

Furthermore, its dual constraints matrix lias a primai block-angular struc-

ture. Therefore, the primai program is amendable to an approk -ich by Ben-

ders' decomposition, whereas the dual program can be challenged by the

Dantzig-Wolfe reformulation and colunm generation techniques. These two

approaches are equivalent by duality.

A Two-Level Interior-Point Decomposition Algorithrn for Multi-Stage Stochas tic Capacity Planning and Technology Acquisition 93

In the present chapter, we look at the stochastic dual program and de-

compose it along the scenarios. Hence, by utilizing the Dantzig-Wolfe re-

formulation, the ACCPM and column generation can solve the three-stage

problem in two levels by way of the following procedure. In the first level, the

&composition is composed of a master program that is solved by the AC-

CPM, and subproblems. Each subproblem &composes into a second-level

master program that is also solved by the ACCPM, and subproblems that,

are solved by the simplex method. Figure 4.1 illustrates the two-level decom-

position along the scenarios for a three-stage stochastic program. The loft

side of Figure 4.1 illustrates the first-level partitioning, whereas the right sicle

shows how a subproblem in the second stage is decomposed into a n'aster

program and its subproblems.

Figure 4.1: Two-level &composition along the scenarios.

To observe how the two-level &composition is donc on the dual program,

keep the first, constraint, AT A + /Tri- < c, in the master (as the complicating

constraint). Thon, assmning that the subproblem [SPI.] is bounded, the

(A, n, p.) cari be expressed as a convex combination of the extreme points


( )1/” 7rp: Pp); P e P:

( A l 7r 711) = 1 7TPJLP) 6P;

t9„ = 1.

Op > 0, p E

Therefore, the master problem at the first level becomes:

[Il Pl : min 4/P = (E, Ap op) + c(Ep 7rp Op) + (Ep tep Op)

s. t. AT („ AP eP) + (Ep 9P) < c, Ep Op -= 1,

Op > 0, 1) e P.

Now in [MM, let a l be the vector of the dual variables on the first set of

constraints and ao be the dual variable associated with the convexity row.

Thus. the subproblem at the first level can be written as:

[SPI] : miii z pi = [cr — A cl i ]À +[c7; — T1 ctilz

s.t. WT r

This subproblem is a linear program that also can be decomposed into a

master program and its subproblems. Hence, the natural linking constraints

would be the first set in [SPI.] that links variables Tc and p. This linking struc-

ture remains in the second master program. Assmning that the subproblem

A Two-Level Interior-Point Decornposition Algorithra for Multi-Stage Stochastic Capacity Planning and Technology Acquisition 95

[S1)11] is bounded, the (A, 7, ft) cari be expressed as a convex combination of

the extreme points (A„,7F q , itg ), q e Q:

(A, Ti;. ( [tg ) 0,1 ,

> 0q = 1,

0,1 > 0, e Q.

Hence, the second master program appears as:

: = min [cr — Aai ] (Eq Ag o-q ) [eTr — T1 a1 ] (E 7i Gr )

-I-C (Eq itg 0-q )

S.t . i4/1. (Eq q 0-q ) (Eq fig o-q ) qi ,

= 1,

0 q >O , q E Q.

Similar to what vas clone above, let a 2 be the vector of dual variables on

the first set of constraints in [MP1] and a, be the dual variable associated

with the convexity row. Thus, the second subproblem appears as:

[SP11] : min = — Aad À + {e,T — T1 ai — Wiaz] ir

+ [et,. — T2 az] [L — aa

s. t. Wr IL < q2.

Note that the Dantzig-Wolfe reformulation is donc in the first and sec-

ond master programs. The second master program is a subproblem for the


first master, providing columns to be dynamically added to the first n'aster

teogram. Moreover, the second subproblem provides columns to be added

to the second master.

4.4 ACCPM

The analytic center cutting plane method (ACCPM) [33] combinas the inte-

rior point algorithm with cutting plane techniques to provide a powerful tool

for solving the large-scale optimization problems in general, and stochastic

programs in particular. The ACCPM bas a superior global convergence prop-

erty and is amendable to decomposition methods. In solving [TSM], we take

advantage of the fact that the convergence rate of the ACCPM is insensi-

tive to the problem scale[38]. This unique property allows us to solve luger

problems while using almost the same number of iterations as the smaller

ones.

As explained in the previous section, we design a two-level decomposition

algorithm for solving a class of multi-stage stochastic programs. Using this

algorithm, WC solve the master problems, namely [MP] and [MP1] with the

ACCPM. The following section describes how the ACCPM is implemented

in the master programs.

4.4.1 Solving the Master Problems

The decomposition principle along with the dynamic colunm generation tech-

nique imposed on the dual of a stochastic program works with a small subset

of columns of the full master programs in [MP] or [MP1]. This subset of

A Two-Level Intcrior-Point Decornposition Algorithin for Multi-Stage Stochas tic Capacity Planning and Technology Acquisition 97

columns forms the restricted master program that in general format eau be

represented as:

ZR* Afp = min{c,Ty • ATy < _

where Ar and cr represent the subset of columns of the full n'aster and their

associated cost vector, respectively. Index r is a particular iteration at the

master program level.

At a given iteration. ACCPM solves a restricted master problem -namely

[Mei associated with [MM, and [RMP11 associated with [MP1]--- to send

the query points (central dual prices) to the oracle (subproblem). hi addi-

tion, the solution of the restricted n'aster program provides a feasible primal

solution to the full master, winch updates the upper bound,, in each level,

for example,271 = mint.7; 1 , zl for the first level or 7-,2 = mire2 , Z *Rdwpd

for the second level program. In return, the oracles [SPI. and [SP 11] generate

columns to be &Med to their corresponding n'aster program. Tire stopping

criteria imposed on the problem measures the solutions of the restricted mas-

ter program against that of the full n'aster at optimality. This means that the

algorithm stops iterating when the restricted n'aster program demonstrates a

solution that is nearly optimal to the full n'aster program. Moreover, the AC-

CPM uses a column generation technique to localize tire polyhedron within

the current relaxed feasible region related to [MM or [MP1], in which their

optimal solutions exist. Such polyhedrons are called localization sets. Based

on that we deline the localization set as:

y > z,A,T. y <

A Two-Level Interior-Point Decomposition Algorithrn for Multi-Stage Stochastic Capacity Planning and Technology Acquisition 98

where z is a lower bound provided by the oracle to the master program, and

is denoted by

z i = max{ z i , +

for the first master program [MM and

2:2 = 11{ 2 , z2 + 4P11}

for the second master program [MP1].

Optiinizing a log-barrier function over the localization set provides an

analytic center and the corresponding dual value. This dual value is sent

to the oracle to generate column Br and its associated cost value dr , winch

in turn is added to the n'aster problem. This action updates the constraint

matrix and the cost vector in the restricted n'aster problem as follows:

4r+1 = [Ar Br]; cr+1 = [cr- dr].

The algorithm terminates when the difference between the lower and upper

bound falls below a predetermined threshold. The basic steps of a generic

ACCPM algorithm eau be summarized as follows:

Algorithm 4.1: ACCPM

Initialization

A feasible point y° and its corresponding lower and upper bounds z, >.=5.

The initial set of localization

Main iterations While lz — >

A Two-Level 1liter-1:or-Point Decomposition Algorithrn for Ilitlti-Stage Stochastic Capacity Planning and Technology Acquisition 99

1. Compute the analytic center of L. say y, and its corresponding dual solution x.

2. Cali the oracle at point x and update the lower and lippe' bounds z,

3. Update the localization set

End—while

Unlike many decomposition methods, the restricted master program in

ACCPM is not solved to optimality. Instead, it seeks the anal tic center

of the localization set as a starting point for the next iteration. Then, tins

analytic center passes forward to the oracle to generate new columns. You eau

calculate an analytical center by the primai Newton method. dual Newton

method, or primai-dual Newton method. In our implementation, we use the

primal Newton method, since no guarantee exists that the dual feasibility cari

be recovered after cutting a portion of the dual space. Note that ACCPM

needs a feasible point to start with at each iteration, and only a primai

feasible point can Le recovered perfectly after adding columns [36, 56].

4.4.2 The Two-level ACCPM

The two-level decomposition method is implemented to solve the multi-stage

stochastic program in the following mariner: ACCPM solves the first stage

and passes forward the current central solution to the later stages, while infor-

mation passes backward in the form of colunms to be added to the ancestral

problems associated with the descendent stages. This can be summarized by

the following pseudo-code.

A Two-Level Interior-Point Decomposition Algorithm for Multi-Stage Stochas tic Capacity Planning and Technology Acquisition 100

Algorithm 4.2: Two-level Column Generation by ACCPM

mit ialization

Starting primal feasible points are y? and A for the first and second n'aster programs. Their corresponding lower and upper bounds are fi and z2 . z2 .

The initial sets of localization for the first and second masters are characterized by = {y]. : b1 Vi > < £2,= {y2 : b2 P2 > Z2 , A2 y2 < c2} where A = A2 = [I, — Vectors are

= (1 and c2 = (12 , u2 )T , where 1 1 , u 1 . 12 , u2 provide bounds on variables yi and y2 for the first and the second masters.

Main iterations While — > e

1.1 Compute the analytic center of say pal , and its corresponding dual solution ra i .

While 1f2 — z2I > f 9.1 Compute the analytic center of .C 22 , say ya2 , and its corresponding

dual solution xa2 ; update the lower bound

9.2 Call the subproblem [SP1 1] at point (ra i , xa2 ). It returns column Bq and its associated cost vector (1q. Update the lower bound Z 2 .

2.3 Update the localization set £ 22 by updating the coefficient matrix = .897 and el dqr•

2.4 Update matrix Dq+ 1 = [Dg Bq], which consists of ail extreme points of subproblem [SP1 1] generated so far.

2.5 -Update the upper bound by using i = min{21 , c2 ya2}.

End—while

1.2 Compute the convex combination of colmnns gathered in D by using the analytic center ya,2 , and generate the column BP and its associated cost rP.

A Two-Level hiterior-Point Decomposition Algorithin for Multi-Stage Stochastic Capacity Planning and Technology Acquisition 101

1.3 Update L by updating the coefficient matrix Ari = BPI' and (,p-Fr ,491T . -1 '

1.4 Update the upper bound Z2 by using Z2 = Min{ yai }.

End-while

Note that in the above algorithm ; we need the convex combination of

ail columns that are generated in the second loop to be passed forward as a

new column to the first loop (the first master problem). Therefore, in 2.4 we

design matrix D to gather ail the columns that are generated so far in the

second loop. In the implementation of Dantzig-Wolfe, each thne we enter the

second loop, we keep the previously generated columns, whereas in ACCPM

matrix D is reset to the empty matrix. In the above pseudo-code. the convex

combination of the colunms that is gathered in D is denoted by B.

4.5 Computation.al Experiments

For the computational experiments, WC implemented the two-level decompo-

sition algorithm on the capacity expansion and technology acquisition model

[TSM]. The code is written in MATLAB version 7.1 for both the two-level

ACCPM and the two-level Dantzig-Wolfe. In the Dantzig-Wolfe decompo-

sition approach, the two linear relaxation of the restricted n'aster problems

and the subproblem are solved using LINPROG, the build-in library within

MATLAB 7.1. Moreover ; in the two-level ACCPM ; the two linear relaxation

of the restricted master problems are solved with an in-house version of AC-

CPM (with a duality gai:) set at 10 -g ), and the subproblem is solved 1 -.)y using

LINPROG. [TSM] proposes a multi-stage stochastic program that models


demand through a SCellari0 tree and can accommodate any discrete demand

distribution with any number of products. We explored this possibility to

create different joint probability distributions of two products demand, so

to be able to compare the two-level ACCPM against the two-level Dantzig-

Wolfe in different dimensions. For example, in the first row of the Table 1,

2-Demand means that we have 2 types of forecasted demand D8), with

probability Vs. s = 1,2. Silice we consider two products, it generates 4 possi-

ble scenarios in the first stage and 16 scenarios in the second stage. Similarly,

the scenarios can be constructed through 3-Demand, 4-Demand, 5-Demand,

and 8-Demand, processes that generate 81, 256, 625, and 4096 scenarios

respectively.

The decomposition and partitioning procedure is (loue on the dual of

[TSM] to take advantage of decomposing the problem along the stages. In

ternis of the number of variables in the dual problem, we solved a range

of problems from 63 variables to 12 483 variables, whereas the number of

constraints ranged from 54 to 8 454. The MATLAB limitations did not allow

us to make the constraint matrix beyond 12 483 x 8 454, which corresponds

to the 8-Demand model. The features of the test problems are presented in

the Table 1:

An inner iteration of both ACCPM and Dantzig-Wolfe (DW) corresponds

to an inner loop of the program in which the second n'aster problem and the

subproblem interact. Alternatively, an outer iteration corresponds to an

outer loop where the first master progTam interacts with the information

received from the inner loop. As is presented in Table 1, the nurnber of outer

iterations in bath ACCPM and DW are comparable for the aforementioned

A Two-Level Interior-Point Decomposition Alqorithm for MUlti-Stage Stochastic Capacity Planning and Technology Acquisition 103

Table 4.1: Computational R.esults

Number of Scenarios

Primai var.

Dual var.

ACCPM DW Inner Outer muer Outer

2-Demand 24 = 16 :54 63 30 5 95 5 3-Demand 34 = 81 204 273 98 4 348 6 4-Demancl 44 = 256 582 819 15 3 709 5 5-Demand 54 = 625 1356 1953 15 3 1939 4 8-Demand 84 = 4096 8454 12483 26 3 7262 3

problem sizes. However ; the real difference is in ternis of the nurnber of inner

iterations, where ACCPM performed much better, especially for the luger

problems. For the 2-Demand problem, the number of outer iterations is the

saine, but less muer iterations occur for the Dantzig-Wolfe decomposition

method. For the last four problems ; the nimber of inner iterations is much

larger for the Dantzig-Wolfe method as compared to ACCPM ; that is, from

10 to almost 300 times.

The first four models in this table were solved on a DELL workstation,

whereas the last model vas submitted to Sun Grid due to the large dimen-

sionality of the problem. Although we did the computing experiments on two

different machines ; in both cases ; ACCPM performed inuch faster in ternis

of the CPU thne. ACCPM solved ail problems in a few minutes. Except

for the first problem, winch the Dantzig-Wolfe approach a,lso solved in a few

minutes ; it took this method hours to solve the rest of the problems.

A Two-Level Interior-Point Decornposition Algorithin for M'alti-Stage Stochas tic Capacity Planning and Technology Acquisition 104

4.6 Conclusion

The improvement of manufacturing flexibility is becoming an increasingly

important method to achieve a competitive advantage in the market place.

Virais are seeking new ways to integrate flexibility in their manufacturing

process to flot only survive in an uncertain environment but to create core

competencies that are absent in their competitors strategies.

Researchers have explored many aspects of flexibilities; however, the

prescrit chapter focuses on capacity, production, and volume flexibility in an

environment that is characterized by variable product demand and multiple

production. We use the multi-stage stochastic program to capture the evolu-

tion of product demand over Urne. Since the multi-stage stochastie program

is a large-scale optimization problem, and in many cases not tractable by

standard optimization packages, we developed a two-level decomposition al-

gorithm combined with a dynamic colurrm generation approach to reduce the

dimensiomdity of the problem at cadi level. The resulting algorithm is irn-

j)lenientecl on the standard Dantzig-Wolfe decomposition method and tuât of

ACCPM. Computational experhnents show that, the two-level interior-point-

based method ACCPM solves the problems in a small number of iterations

as compared to that of the simplex-based Dantzig-Wolfe method.

Chapter 5

Solving Multi-Stage Stochastic,

In-house Production and

Outsourcing Planning by

Two-level Decomposition

5.1 Introduction

In the past few decades, manufacturers have paid increasing attention to out-

sourcing products and processes. Traditionally, outsourcing by organisations

vas clone to reduce the marginal production cost and to exploite the exter-

nal supplier's capabilities. However, in many cases a number of important

factors such as product quality, lead tulle rehability. technical capability ; and

the financial sta,bility of the supplier were flot taken into accolait. Many com-

parlies decided to outsource rather than make other potential decisions based

Solving Multi-Stage Stocho„9tic, In-house Production and Outsourcing Planning by Two-level Decornposition 106

on a short terni analysis of cost reduction and capacity [12]. This approach

may have resulted in high contingency costs for corporations. For example, a

recent massive toy recall cost Mattel nearly 30 million U.S. dollars [39]. The

recall was made alter Chinese-manufactured toys were found to have paint

with excessive amounts of lead or small parts that could be swallowed by

children.

In 1988, after comprehensive research vas carried out by McKinsey and

Company [49], Mattel started a Vendor Operation Asia (VO.A). The research

suggests that Matte' manufactures the core products like Barbie and Hot

Wheels in-house and outsources ail non-core products. By using this strat-

egy, Mattel eau produce a large number of toys with short life-cycles without

any intensive investment. Over recent years, Mattel has introduced over 300

new, non-core toys each year. The vendors are working with Mattel un-

der an agreement called Original Equipment Manufacturing (OEM) [40] by

which l\lattel transfers the design and knowledge to vendors in China, and

sells the final products under its own brand naine. The recall of millions of

toys by such a giant corporation, as well as recalls on other unsafe products-

including tainted pet food, seafood for human consumption, and toothpaste-

indicate that quality failure is a large risk assoeiated with outsourcing to

overseas suppliers, especially when the vendors have no stake in the brand

naine. Moreover, product safety standards can be improved as the existing

infrastructure improves in a country, since what is considered to be unsafe by

Western standards may be perceived as sale in developing countries. There-

fore, Western importers should benchmark a safety standard and monitor

the quality during the manufacturing process. By doing Ulis, a corporation

Solving Multi-Stage Stochustic, In-hou,se Production and Outsourcing Planning by Two-level Decornposition 107

like Mattel could have stopped or improved the manufacturing process before

20 million unsafe toys were introduced into the market. However, the extra

cost associated with quality assurance may prevent many corporation from

participating in a consistent quality monitoring.

Considering these challenging issues, in recent years, most manufacturers

have addressed the question of how extensively they should outsource, and

at what point should they stop? How cari manufacturers assess the risks of

making or buying to best leverage their resources for increased profitability?

What is the optimal combination of make and bu?

To help to answer these questions, we developed a multi-stage stochastic

programming model which takes into account the uncertainty involved in

the production of the quality of outsourced products in the face of stochastic

demand. The goal is to find the optimal way to choose between in-house

capa.city expansion and buying from the local suppliers with assured quality

versus buying from overseas suppliers.

Mufti-stage stochastic programs (MSPs) are large-scale optimization prob-

lems. The size of MSPs constraint matrix increases exponentially with the

number of stages and the 'nimber of scenarios. Hence, soine MSPs are in-

tractable by direct optimization solvers, and decomposition methods become

good alternatives for solving them. However, in classical decomposition ap-

proaches, the number of iterations eau become unmanageable, especially as

the dimension of the n'aster problem increases. To resolve this issue, we pro-

posed in [62] a two-level interior point decomposition method for solving

class of multi-stage stochastic programs with recourse. This two-level decom-

position method uses dynamic column generation and consists of the solution

Solving Multi-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decomposition 108

of the two master programs and a subproblem. Both master programs have

Dantzig-Wolfe reformulations that are solved by an interior point-based de-

composition algorithm, such as the Analytic Center Cutting Plane Method

(ACCPM) (sec [33]) to provide central dual prices. The coinbination of the

second n'aster program and its subproblem acts as a subproblem for the first

n'aster. The ACCPM has a superior global convergence property, and its

convergence rate is insensitive to the problem scale. These unique properties

allows us to solve large-scale MSPs in a manageable number of iterations.

In this chapter, we take adwmtage of the recourse structure of the in-house

production and outsourcing planning mode! to &compose and solve it with a

state-of-the-art two-level decomposition algorithm. We extend the two-level

decomposition via the ACCPM to Benders' decomposition for the first time

so to be able to compare the computational results with the two-level column

generation that uses the ACCPM. Moreover, to achieve a higher accuracy,

we implemented a two-level decomposition algorithin where the first level is

solved via the classical Dantzig-Wolfe &composition and the second level is

solved via the ACCPM.

The present chapter is organized as follows. In Section 2, we formulate

the in-house production and outsoursing planning problem. In Section 3, we

explain the approaches to solving the problem, that is, a two-level column

generation and a two-level Benders' decomposition, and an interior point-

based cutting plane algorithm — the ACCPM. In Section 5, we discuss the

computational experiments. Finally, in Section 6, we present oui concluding

remarks.


5.2 In-House Production/Outsourcing Plan-

ning Mode!

As product demand increases, corporations face decisions about where and

how to expand their production capacities. Many business organizations

consider outsourcing their products to the emerging, low-cost overseas mar-

ket. Outsourcing allows corporations to meet customer demand at lower

marginal costs, while also maintaining flexibility of °pendions in changing

environments. However, the real evaluation of the cost-savings associated

with outsourcing requires more than a mere consideration of the production

cost differences. Many manufacturers face a 'nimber of costly, outsourcing

challenges including lead times, quality, and communication. To capture the

real cost associated with outsourcing, we use the scenario tree approach to

model the evolution of the uncertain demand and risk associated with the

quality of outsourced products in multiple periods. Elements of the model

are illustrated in Figure 1.

This figure shows that in each stage the decision ma,kers face two alterna-

tive paradigms. The first, paradigm, which is illustrated in the upper part of

the decision tree, is concerned -with buying products of uncertain quality from

overseas suppliers. This eau be explained if, for example, we assume three

different categories associated with quality — namely, E (Excellent quality),

A (Acceptable quality), and B (Bad quality). Shnilarly, we assume three

other categories for demand — namely, H (High), M (Medium), and L (Low).

Therefore, the first scenario EH in the upper part of the tree shows the pos-

sibility of having excellent quality and high demand. On the other hand, in


Figure 5.1: In-house production and outsourcing planning, decision tree

the lower part of the tree, for manufacturers facing the decision of whether to

expand in-house capacity or to buy from local suppliers with assured quality

and higher cost, the only stochastic parameter is the demand. The goal of

the proposed model is to find the optimal way to choose between in-house

production, buying from local suppliers with higher cost, or buying from low-

cost overseas suppliers while taking into account the risk associated with the

quality level. In this chapter, we refer to the upper scenarios as the first set

of scenarios and to those in the lower part as the second set of scenarios.

The first set lias two parts starting from stage 2 as illustrated in Figure

5.1; the number of scenarios in the upper part is denoted by Kt and the

number in the lower part it is denoted by J. Similarly, the 'nimber of

scenarios in the lower part is denoted across the planning horizon t.

Solving Multi-Stage Stochastic, In-hanse Production and Outsourcing Plan,ning by Two-level Decomposition 111

The indices, variables, and input parameters used in the model are as

follows:

Indices: T: number of stages (periods) in the planning horizon Kt : number of first set scenarios (concerned with outsource plan-

ning) in stage t. t =1, • • • ,T Jî: number of first set scenarios (concerned with buying from

local suppliers) in stage t, t = 2. • • • . T Jt : number of second set of scenarios in stage t, t = 1. • • • ,T oi(kt ): capacity index for scenario

capacity index for scenario j t clUn: capa.city index for scenario

X0:

tke:

Yo: Yti?:

1/ki t :

Ztkt:

Wti?:

Vti?:

Vi • •

Utkt :

U .2. t../t •

Utit :

Solving Multi-Stage Stochastic, In-house Production and Outsourcing Planning by Tivo-level Decomposition 112

Variables: amount of product winch is manufactured in stage 0, amount of product which is manufactured in stage t,under scenario k1 . t =1. • • ,T, 1c1 = 1, • • amount of product bought from local suppliers in stage 0, amount of product bought from local suppliers in stage t under scenario jt2 , t = 1, • • • T, jt2 = 2, • • • Jt2 amount of procluct bought from local suppliers in stage t under scenario j t , t = 1, • • T, j = 1, • , jt amount of product which is outsom•ced at stage t under scenario kt , t =1,• • ,T, kt = 1, • • , Kt the amount of product which is manufactured in stage t, under scenario kt , t = T. kt = 1, • • ; Kt the amount of product which is manufactured in stage t, under scenario j. t = 1, • • • ,T, jï =1,• • • amollie of product bought from local suppliers at stage t and for scenario jî, t = 1, • • T, jt2 = • • ft2 amount of product bought from local suppliers at stage t ami for scenario jt , t = 1, • • • T, t= 1. • • • Jt additional capacity at the begirming of stage 0 additional capacity at the beginning of stage t, 'Eider scenario

t =1, • • ,T, kt = 1,- • • , additional capacity at the beginning of stage t, under scenario

t = 1 , • • • , T, = 1, • - • , additional capacity at the beginning of stage t, under scenario it2 , t = 1 , • • • T, it = 1 , • • • , Jt.

Solving Multi-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decomposition

Parameters:

Uo: initial capacity at stage 0

Co: marginal cost of production at stage 0 etkt: marginal cost of production at stage t -ander scenario k1 , t =

1,• • • ,T, kt =1,- , Kt Cdo: unit buying cost from local suppliers at stage 0 Stkt : unit cost for outsourcing at stage t, under scenario kt

ePtkt : unit cost of in-house capacity expansion at stage t, under

scenario k1 , t = 1, • • • ; 71 , = 1, • • Kt CPti?: unit cost of in-bouse capacity expansion at stage t, under

scenario j?, t 1, • • , T, j? = 1. • • • cpti,: unit cost of in-bouse capaeity expansion at stage t, under

scenario Lt, t = 1, • - • T. Lit =1,- • unit production cost at stage t, ululer scenario j?, t = 1, • • • 7", = 1, • • •

atit : unit production cost at stage t, under scenario jt , t =1,- •• ,T, :ft = 1,- • •

b. 2 •

unit buying cost from local suppliers at stage t, under scenario t

1, • • • ,T, ft2 = 1, • ft2

btit: unit. buying cost from local suppliers at stage t,

under scenario Lit, t = 1,• • ,T, Lit =-1,• • „It dtkt : unit demand at stage t, under scenario kt , t = 1, • • • T, kt =

1, • • • , Kt dti?: unit dernand at stage t, under scenario t t =1,- • • ,T,

1, • , dtit : -unit demand at stage t, under scenario Lit, t = 1, • • • T, jt =

1, • , ,It Ptkt: probability of scenario kt in stage t, t = 1, • ••,T, kt =

1, • • , Kt Ptj;2 probability of scenario .9 in stage t, t = 1, • • , T, =

1, • • • P. probability of scenario Lit in stage t, t = 1, • • • T, Lit =

1, ••• , Jt

Solving Multi-Stage Stochastic, In-house Production and Outsouming Planning by Two-level Decomposition 114

Given the above definitions, the general T-stage model can be written as

follows:

[TA1B] : max (c0 X0 + cd 0 Y0) + T Kt

E c.Ptk Utk t=1 k=1

ePti rjtj

T

T Kt

cPti 2 Utj 2 +

Ptkt ( Cikt tkt Stkt Ztkt) t=1 j2 =1

T

E E Pti? t=1 .•2_1 - T Jt

t=1 k=1

(ati? Vti? + bti? Vii?)

Pti, (atit + hti, Vtj t )

Solving Multi-Stag e Stochastic, Iii-hanse Production and Outsourcing Planning by Two-level &composition 115

s. t. X0 < rio + U0, (5.1)

.X0+ Yo > do , (5.2) T cx(Kt )

Xtkt 5_ U 0 + Uo + E ut„,k, ) , ck(À:t) = 1. • • • , Kt, t=1 k=1

T ct(it)

1 4, <Po + + tet(jt), Cit ) = 1, • ,

t=1 k=1

i'Vt3? 5_ 1 o E t=2

X tk, Ztk t > dtkt ,

Wij? Vti? ?_ dt

Wt.it >

X, Z, U, W, > O.

T

(5.4)

Ute,u2), a(e) =- 1, • •

k=1

(5.5)

= 1, A 1 , t = 1, T

(5.6)

jt = 1„ • • . t = 2, ..,T 2

(5.7)

• Jt . t = 1, ..,T

(5.8)

(5.9)

In [TM13], constraint (5.1) refers to the capacity constraint in stage 0,

while constraint (5.2) specifies that the dernand must be satisfied in stage O.

Solving Multi-Stage Stochastic, In-ho use Production and Outsourcing Planning by Two-level Decornposition 116

In stages t = 1, ... T. constraint (5.3) represents the capa.city restriction on

production when a. manufacturer considers buying from overseas suppliers

at a lower price and risky quality. Constraint (5.4) represents the capacity

restriction on production when manufacturers also have the choice to buy

from local suppliers at a higher price ami assured quality. ln stages fi = 2, ...

T, constraints (5.5) refers to the capacity constraint when buying from local

suppliers (in the upper part of Figure 1). Constraints (5.6). (5.7) and (5.8)

force the demand to be satistled in both situations —namely, to outsource to

overseas suppliers or to buy from local suppliers.

5.3 Solution Approaches

Decomposition approaches become methods of choice for solving large-scale

MSP problems. They can reduce the dimensionality of the large-scale prob-

lems, since the y are amendable to a parallel computer environment. The

well-known nested decomposition method [6] for solving MSPs cari be impie-

mented via parallel computing, sec [65]. By parallelization, sets of scenarios

are assigned to processors to create cuts (columns) simultaneously, and thus,

problems are solved more quickly.

The simplex-based decomposition approaches are prevalently utilized for

solving MSP problems. However ; the use of an interior-point method is often

better choke for solving large-scale optimization problems. In [62]. we SUC-

cessfully developed a two-level interior-point-based &composition algorithm

for a class of MSP problems. In the present chapter, we use variants of the

two-level approach to solve the [TMB]. In addition, we implement a new


two-level decomposition based on the Benders decomposition of the [TMB].

This strategy allows US to propose the best approach for solving the in-hand

problem.

5.3.1 A Two-level Column Generation Approach

To best explain the Dantzig-Wolfe decomposition [54] and the column geler-

ation approach that is implemented on [TM13] in two levels , we formulate a

generic form of [TMB] in three stages. This formulation consists of a deter-

ministic stage (stage 0) and two other stages with stochastic variables, where

the first stage lias ru scenarios and the second stage lias Tt --= et' scenarios.

In this three-stage model, w is the vector of variables associated with stage

0 ; whereas yi = {iA ; W, • ; } and 1/2 ={y?, y.. ; y} are the vectors of

variables associated with the first, stage and the second stage, respectively.

The cost structure consists of the direct cost e of stage 0 ; and the expected

costs of the stochastic variables in the first and second stages, denoted by

and q2 , respectively. Then, the generic primai three-stage stochastic program

can be formulated as follows:

[Primat] : max eTx +qiyi +(h y2

=b

+ 13/4y1 = //1

+ nyi +1472 y2 = /12

Solving Adulti-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decompositio 72, 118

To case the notation. we rename the right-hand side of the above model as:

b = cx, li = c.„ and h9 = c. Then, the associated generic dual problem is

formulated as the following:

[Dual] : max c7; À +C,TTC +CiDt

s.t. AT +TiTz +BT/t <

1/1"T7r +TT/c. < qi

< q2

where À. r arid /2 are the dual vectors associated with the primai constraints

of stages zero, one, and two, respectively.

The decomposition along the scenarios is donc on the dual program by

using the column generation technique and is explained in this section. More-

over, the decomposition along the stages in the scenario free, donc on the

primai program by utilizing Bender's cuits, is the topic for the next section.

Although these &compositions are equivalent by duality, they generate dif-

ferent computational results in terms of the number of iterations and the

precision of the algorithms.

To best exploit the structure of the dual program let us keep the first

dual constraint, A T À + TiTz + BT /.2 < c, in the first master as the compli-

cating constraint. Then, assuming that the subproblem [SPI.] is bounded,

the (À, ir . )1.0 cari be expressed as a convex combination of the extreme points

Solving 114ulti-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decomposition 119

(4, 7rp, fip), P e P:

( À ' =

E op Op > O. p E P.

Therefore, the master problem at the first level becomes:

[111P] : min z*mp = (Ep Ap Op) ± CI; (II, 7Fp Op) + c(Ep PpOp )

s. t. AT (Ep Ap Op) ± (Ep '7p Op) ± BT (Ep P p Op) < C,

Ep Op = 1.

Op > O. P E P.

Now in [MP], let a i be the vector of the dual variables on the first set of

constraints and (19 be the dual variable associated with the convexity row.

'nus, the subproblem at the first level can be written as:

[SP1] : max z, 1 = — A a i jA + [cT — ai fr + [ct7; — B ai ] p — ao

S .t 1/17T TC

+r21 1-1 qi ,

< (12.

Assuming that the subproblem [S13 11} defined below is bounded, the vec-

tor (A,7r,p) can also be expressed as a convex combination of the extreme

Solving Multi-Stage Stochastic. In-house Production and Outsourcing Planning by Two-level Decomposition 120

points (Aq , Trq , [tg ) ; q G (2:

=

jj 0„ -= 1.

Oq > 0. (J E Q.

Hence, the solution of [S1)1] eau be replaced by an iterative process by using

a second n'aster program [MP1] and its column generator [SPI 1]:

[M Pl] : 'A1P1 MaX [cl Ct (E q q) [CT ad (Eq rcq o-q )

+ (ct,T — B ct ij (Eq aq )

s. t. (Eq zq o-q ) + T27 (Eq p q

Eq a,=1, a >0 g e Q.

—

<

Similar to what vas clone above, let a 2 be the vector of dual variables on the

first set of constraints in [MPI.] and a, be the dual variable associated with

the convexity row. Thus, the subproblem is given by:

[SP11] : max = — -4c/1iA + [c„T — T1 c — I'V1ct217

+ - T2 a2 — B a2 jp. —

s.t.

At any given iteration, bounds on the optimal value of the master pro-

Solving Multi-Stage Stocha,stic, In-house Production and Outsourcing Planning by Two-level Decomposition 121

gTams are updated. The lower bound on the first master program [IVIP] is

computed as:

Zmp = IrlaX{,?_;mp, -Z-mp Zs* pi} = MaX1,1„mp, -27mp ZMp l

while, for the second master program [MP1], it is given by

zmpi = max{_&mpi, 72 4- z*.spn

The master programs are initially defined on a small subset of columns of

the full n'aster programs, and the former are called restricted n'aster pro-

grams. We denote the restricted n'asters by [RMP] and [IIMP1] for the first

ami second master programs, respectively. The upper bounds are updated

by the current optimal solution to the restricted master programs and given

by 1vIP = Z*Riti p and 7111 Pi = 4111P1 . In each love!. the iterations termi-

nate when the difference between the upper and lower bounds falls below a

predetermined threshold. A pseudo-code for the proposed two-level column

generation approach is presented in Algorithm 5.1.

Algorithm 5.1: Two-level ColuTrif i Generation Algorithin

Initialization Feasible or artificial columns for [R1\413]; lower and upper bounds (z mp ,

Outer iterations While Ifmp — zmp i > Emp

1.1 Solve the restricted n'aster problem [RMP]; compute primal variables Op; p e P, and dual variables (ai, ae).

Solving Multi-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decoraposition 122

Inner iterations Feasible or artificial columns for [RMP1]; lowers and upper bounds (z mpl , .z7mpi)•

While > emPi

2.1 Solve the restricted master problem [RMP1] at point (ai, a0); compute primai variables 61q . q E Q. and dual variables (a2, au).

2.2 Cal! subproblem [SPB11] at point (a l , a0.a2 , au ); generate extreme point (A g , 7rq , mq ), q E Q. and its associated cost

dq .

2.3 Update [RMP1] using extreme point (Ag; 7r q : itg ) of cost dq .

2.4 Update lower and lippe' . bounds (zmpi ,

End—while

1.2 Generate extreme point (4, ir , pp) = 7,, mg ) 61q , for p e P. the convex combination of columns in [R.MP1], and its cost rp .

1.3 Update [R,MI1 using extreme point (4. 7Tp, tip) of cost rp .

1.4 Update lower and upper bounds (2, mp , fiwp).

End—while

In the inner iterations, given the extreme point (A g . 71- flq ) , for q G Q,

the constraint matrix of R.MP1 is updated by adding a new column, 14/TIr q +

TI ft which is independent of Ag . The cost (.4 of this new column is computed

IS

de = [c7; Aa l'Ag + [C,T — T1 1 clip— ( f + T B

Similarly, in the outer iterations. given 7rp , pp ) = Eq (Aq , tiq ) 0q , for

p E P, the constraint matrix of R,MP is updated by aclding the generated


column AT Ap Inrp BT itp The cost rp of this colmnn is given by

T T P + Cr fi P -I- P . P

5.3.2 A Two-level Benders' Decomposition

The Benders' decomposition principle is based on a variable partitioning

procedure in a large-scale primai problem, sec [4]. The method partitions

the variables by fixing the so-called complicating variables. This reduces the

given problem to a more tractable, a:nd hence casier, problem to solve. The

latter problem is parameterized in ternis of the vector of the complicating

variables. The dual to this problem forms the subproblem. In each iteration,

the subproblem passes forward the current dual solution (for a fixed vector

of complicating variables) to a n'aster program in the form of a Benders' cut

that utilizes the linear programming duality to refine the value of the vector

of complicating variables.

In the particular case of MSP problems, the partitioning would be in terms

of the variables in cadi stage. Similar to what we implemented on the dual

problem when we used the column generation technique, we decompose the

generic primai three-stage stochastic problem in two levels by using Benders'

decomposition. This can be donc by first considering the vector of the first-

stage variables - namely yi = { • • , yn1 ,} as the complicating or fixed

Solving Multi-Stage Stocha„stic, In-house Production and Outsourcing Planning by Two-level Decomposition 124

variables. This forms the following subproblem, given that y i is fixed at

[SPB1(y1 )]: zs* pm(yi) = min cTx +q2y2

s.t. A x = cA

Tlx = c„. — 1471

B x ±W2 il2 = (111 T2Th.

Y2> O.

Solving [SPB1(11)] provides an upper bound on the original problem (ass-

suming that this subproblem is bounded). Therefore, the solution to the

original problem consists in solving the above subproblem for the best

value, and it is given by the solution of the following (outer) master prob-

lem:

PlIPB1 : min yi>0

,* --SPB1(Y1)'

Note that the dual feasibility of the subproblem is independent of the value

Th. Let A, 7r, and p. be the dual vectors associated with the primai constraints

of stages zero, one, and two, respeetively. Then, the dual constraints of

[SPB1(ïj 1 )] are:

AT 1-BT1t < c ;

14772 < q2 •

The dual solution of [SPB1(7j1 )1 is given by an extreme point (4, Irp, itp).

Solving Multi-Stage Stochastic, In-housc Production and Outsourcing Planning by Two-level Decomposition 125

p E P. for which the objective function takes value

Ap + (c7, — WlYiPrp + (cf, —112Y0iip + (1.171.

Therefore, the optimal solution to the subproblem eau be interpreted in ternis

of the best extreme point in P. that is:

* '''SPB1 (y-) = rn aXp e P Àp 11/1 )7rp + (c )fip + 1

min el s.t. el> c),Ap + (c7, — 1411 yi)7rp + (cp. — + giF1, P G P.

The outer master problem, which provides a solution to the original problem,

can now be written as follows in ternis of the extreme points in P and the

vector of complicating variables

P 131 : 211 p B = inji e

s. t. el> c), + — 14/1 )7rp + T27Milp P E P

> O.

To complete the two-level Benders' decomposition, note that problem

[SPB1(y1 )] eau ais° be solved by Benders' decomposition by taking the vec-

tor of the second-stage variables y 2 = { y?, A, • • • as the complicating

variables. Therefore, given that y 2 is fixed at. Tj2 , the second subproblem

Solving Mulfi-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decomposition 126

appears as:

[SPB11(p1 , F2 )1 min T ---'s* PBmp,,p2 ) = (.T2.(.T2. + (12T/2

s. t. A x = na; =

— P1

B X = — T27/1 — 14/2

X >0.

Solving [SPB11(1h, y2 )] provides an upper bound on problem [SPB1(7/1 )]

(assuming that the second subproblem is bounded). Therefore, the solution

to the first subproblem given Th consists in solving the above second sub-

problem for the best y2 value, and it is given by the solution of the following

inner master problem:

[MPB1(Th )] : min Y2 >O 2"S* ,y2)"

The set of dual constraints for [SPB11(Th, Th) appears as:

AT +T;rn +/3T < c.

The dual solution of [SPB11(Th,F2 ) is given by an extreme point (Aq , 7rq , tg ),

E Q, for which the objective function takes value

c), Ag + (c7, — Wl Th ) 7fq + (cm — Teh — W2 Y2)/1q + qi Vi + q2

Therefore, the optimal solution to the second subproblem eau be interpreted


in terms of the best extreme point in Q, that is:

P B 11 (Y1,2)

= 111aXgEQ C),Aq (C7 — T471 1) 7r — T2731 — W2 Y2)11q (11 7:1 + g2 F2

= min e 2 s.t. e2> c,,Aq + (cff — yl)7rq + (c, — T2/ — W2 Y2) ibq (11 Ti 1 + Y2 Y2 q E Q.

The inner n'aster problem, which provides a solution to problem [SPB1(7a,

cari now be written as follows in ternis of the extreme points in Q and the

vector of complicating variables 712 :

[iVIPB 1 (Ti1)] • -mpEn(ei ) = min 92

S.t. e2> C),Aq (c, — W1 Y1) 11-9 4- ( cg — 212Y1 — 1'V2 Y2)/tq

±Y1 ± Y2 Y2 • E

Y2 > O.

In Benders' decomposition, the n'aster programs are initially defined on

small subset of cuts, and they are called relaxed n'aster programs. .We

denote the relaxed n'asters by [rMPB] and [rMPB1(Tj1 )] for the outer and

inner master programs, respectively. Moreover, in the two-level Benders' de-

composition, an miter iteration refers to solving [rMPB] where the algorithm

refines the value of the complicating vector y i of the first stage. Alternatively,

amimuer iteration refers to solving [rMPB1(F 1 )] and, given a fixecl value

it refines the value of the complicating vector y2 of the second stage.


To case the notation, in the outer iterations, we &Ilote the solution to

[rIVIPB1(F 1 )] as ;92 and the optimal solution to [MPB1(77 1 )] as y, at the end of

the iterative process. A eut whieh is added to [riVIPBieV] can be represented

as e2 > Bq y2 + dg , where Bq is the constraint rnatrix associated with the

vector of variables y 2 , and dg is the remaining constant term. To generate

this eut, we use the dual extreme point that is eomputed in [SPB11(fh,9-2 )].

Alternatively, in the outer iterations, we denote the solution to [rMPBI as9 1

and the optimal solution to [MPB] as y at the end of the iterative process.

A eut which is added to [rMPB] can be represented as e 1 > Bp +4. where

Bp is the eonstraint matrix associated with the vector of variables y i and dp

is the constant term. This eut is generated by using the dual extreme point

computed in solving [SPB11(Tji ,y)], where y is the optimal solution of the

inner loop for a given value of rh. (Note that this dual extreme point cari

be computed at the last iteration of the muer loop beforc the algorithm gets

into the outer loop.)

In both irmer and outer iteration loops, bounds on the optimal value of

the master programs are updated. The lower bounds are updated by the

current optimal solution of the relaxed master programs and are given by

MPB and ,.z_mpsicgo zi*.mpsnpi ) = erMPB . Moreover, the lipper bound on

the outer master program [MPB] is computed as:

m p g = Mill fi mpg Z - IL* 1PB1(pi)} = ZS* PB11( 1 ,y; )1

while, for the inner master program [MPB1(M, it is given by:

= min{-7.A/Pm(rii) , Z.S* PB11(ui,-2)}.

Solving Multi-Stage Stocho„stic, In-house Production and Outsourcing Planning by Two-level Decomposition 129

The iterations terminate when the differenee between the upper and lower

bounds falls bellow a predetermined threshold. A pseudo-code for the pro-

posed two-level Benders &composition is presented in Algorithm 5.2. For

performance comparisons, we have also implemented a single love! Benders'

&composition where the complicating variables are both Yi and y2 • The

subproblem turns out to be SPB11(91 , 92) while the master problem simul-

taneously optimizes the y-variables of both stages.

Algorithm 5.2: Two-level Benders' Decomposition Alyorithrn

Initialization initial value 71 ;initial lower and upper bounds (zmpB , f'mpB ).

Outer iterations While IzmpB - zmpB i > Emps

Inner iterations Initial value 72 ; lowe.r and upper bounds (zmpin(y,), ZmpRiw,))•

While If-A/p./31( 1 ) _2•Afp./31( 1 )1 > EMPB1

2.1 Cadi subproblem [S P B11(Th ,T j2 )]; eompute the dual extreme point (Ag, 7rq, [tg ), q e Q.

2.2 Update [7-MPB1(7 1 )] with eut "02 > Bq y2 + dg .

2.3 Solve [ritIPB1(71 )]; update the value of 92 .

9 .4 'Update the lower and upper bounds fmpin om).

End-while

1.1 Set y2* := v2 • 1.2 Call subproblem [SPB11(71 , y;)];

compute the dual extreme point (A p , np ,i.tp ),p E P.

1.3 Update [r1\413B] with eut 9 1 > Bp dp.


1.4 Solve [r1\413B]; update the value of

1.5 Update the lower and lippe'. bounds (zmpB ,

End-while

Set y1 Solve [SPB11(e y)] to complete the optimal prima:1 solution in terms of x*.

5.3.3 The Analytic Center Cutting Plane Method

The analytic center cutting plane method can be viewed as an interior-point

decomposition algorithm; see [33]. This method provides a fast algorithm for

solving large-scale optimization problems by mixing an interior-point algo-

rithm and cutting planes. To solve a multi-stage stochastic program with the

ACCPM, one needs to define an oracle and a query point generator. The spe-

cial recourse structure in [TMB] is best exploited by a decomposition method

such as the .ACCPM. This special structure is an aid in designing the two-

level ACCPM by having the query point generators in two levels, namely -

the first master progra,m UMP] for colunm generation and the outer mas-

ter program [MPBI for Benders) and the second master program UMP11 for

column generation and the muer master program [MPB1(ij 1 )] for Benders).

The oracles are also defined as subproblems [SP11] for column generation and

[SPB11(f 1 , 92 )} for Benders. Moreover, this recourse structure forms a dual

block angular structure in [TM13], and a prima! block angular structure in

its dual. These structural advantages allocate the soultion of the dual prob-

lem to the Dantzig-Wolfe reformulation and coluinn generation technique.

Equivalently, the primal cari be tackled by Benders' decomposition. In the

prescrit chapter, we implemented both decomposition methods on [TM13] and

Solving Multi-Stage Stocho„stic, In-house Production and Outsourcing Planning by Two-level Decomposition 131

its dual The two-level column generation via ACCPM, denoted by C:AC-AC,

is implemented in such a procedure where the ACCPM solves both master

programs and forwards the central dual prices to the subproblem. In return,

the subproblem generates colunms to be added to the second level master

program. The pseudo-code for this algorithm appeared in [62]. This decom-

position by colurnn generation on the dual program can alternatively be clone

as C:DW-AC where the first level master program is solved by the classical

Dantzig-Wolfe decomposition, and the second level master program is solved

by the ACCPM. Equivalently, the two-level Benders' decomposition via AC-

CPM, denoted by B:AC-AC, is implemented in the following mariner: the

ACCPM solves both master programs and reflues the value of the vector of

the complicating variables in cadi level by the information feedback given by

the subproblem.

5.4 Comput ational Results

For the computational experiments, we implemented the two-level decom-

position algorithms on the in-house production and outsourcing planning

[TSM]. The code is written in MATLAB version 7.1 for C:AC-AC and C:DW-

AC by using two-level column generation. In the C:DW-AC approach, the

first linear relaxation of the restricted ma.ster problems is solved using LIN-

PROG, the built-in library within MATLAB 7.1, and the second linear re-

laxation of the restricted master problems is solved by the in-house version of

ACCPM. Moreover, in both the two-level column generation approaches, the

subproblem is solved by using LINPROG. Similarly, the two-level Benders'


decomposition B:AC-AC is donc by solving both n'aster progrmns by using

the ACCPM and the subproblem by using LINPROG.

To compare the success of the aforementioned approaches in solving

[TMB], we generate problems based on a different discrete probability distri-

bution of demand and quality. For example, in Problem 1, we have 3 types

of forecasted demand and 3 types of forecasted quality, making 9 scenarios in

the first set of decisions, and 3 scenarios in the second set of decisions in stage

1. In stage 2, this cornes to a total of 117 scenarios for first and second sets

of decisions. Similarly, we have 4 types of demand and 3 types of quality in

Problem 2, 8 types of deinand and 3 types of quality in Problem 3, 10 types

of demand and 5 types of quality in Problem 4, and 13 types of demand and

5 types of quality in Problem 5. The features of the test problems appear in

Table 1.

Table 5.1: Problem Sizes in the Column Generation Approach

Number of Scenarios

Primai var.

Dual var.

Constraints in MP

Constraints in MP1

Constraints in SP11

1 117 274 260 3 36 235 2 208 468 450 3 48 417 3 832 1764 1730 3 96 1665 4 3100 6384 6322 3 130 6251 5 5239 10716 10636 3 234 10479

Note that in the colunm generation approach, an additional constraint is

added to the -first and second master programs as a convexity row. In [62], we

showed that the two-level ACCPM (C:AC-AC) outperformed the two-level

DW (C:DW-DW), where in larger size problems the number of iterations of

Solving Multi-Stage Stochastic, In-house Production and Outsourcing Planning by Two-level Decornposition 133

the C:DW-DW approach was 300 times greater as compared to C:AC-AC.

This motivated us to use C:AC-AC in solving the [TME3]; the computational

results are given in Table 2.

Table 5.2: Computational Results for C:AC-AC

Outer-Gap Inner-Gap Outer-iter Inner-iter 1 le-5 le- 5 4 17 9 le-5 le-4 4 16 3 le-5 le-5 4 22 4 le-3 le-5 2 17 5 le-4 le-5 3 18

This table presents the number of outer and muer iterations required to

reach the optimal solution, along with its relative precision for each problem

size. Note that an inner iteration of C:AC-AC corresponds to an inner loop

of the program in winch the second master program and the subproblem

interact. Alternatively, an outer iteration corresponds to an outer loop where

the first master program interacts with the information received from the

biner loop. The relative precision for the outer and inner iterations in C:AC-

AC barely reaches to 10-5 ; beyond that, it takes the ACCPM hundreds or

even thousands of iterations to reach the optimal solution with 10 -6 relative

precision. Although ACCPM has an extremely good convergence rate, its

relative precision is not always satisfactory, as mentioned in [2]. The recent

attempts try to improve the algorithm by using the column (cut) elimination

[2]. In [32], it is indicated that there is a "trade-off between required relative

precision of optimum and time needed to reach it." In solving [TMI3], we

explore this trade-off so to reach its optimal solution with higher precision.


Since [TMB] lias a sinall first master problem, we solved the first master

program by classical Dantzig-Wolfe decomposition, using LINPR.OG with its

relative accuracy of 10 -8 ; whereas, we used ACCPM to solve the larger-sized

second n'aster program. The results of this experiment are shown in Table 3.

Table 5.3: Computational Results for C:D'W-AC

Outer-Gap Inner-Gap Outer-iter Inner-iter 1 le-8 le-6 4 21 9 le-8 le-6 5 27 3 le-8 le-5 5 34 4 le-8 le-5 5 29 5 le-8 le-5 6 27

A comparison of the results in Table 5.2 and Table 5.3 indicates that the

relative precision for both inner and outer gaps are improved in C:DW-AC

as compared to C:AC-AC. The number of outer iterations stayed relatively

the same, but the number of inner iterations are larget. for C:DW-AC. The

C:DW-AC reaches the optimal solution with an extremely good precision

and a small number of iterations.

Finally, we implemented the two-level ACCPM via Benders' cuts on

[M131 The computational results for this third decomposition method B:AC-

AC are shown in Table 5.4.

In Table 5.4, outer iteration refers to solving the first master program

by ACCPM where the algorithm refines the value of the complicating vector

of the first stage. Alternatively, mimer iteration refers to solving the sec-

ond master by ACCPM and refining the value of the complicating vector

in the second stage. The aforemention computational results show that the


Table 5.4: Computational R,esults for B:AC-AC

Outer-Gap muer-Cap Outer-iter Inner-iter 1 le-2 le- 3 6 5 2 le-1 le-2 7 7 3 le-1 le-2 5 4 4 - - - - 5 - - -

two-level Benders' decomposition failed for the larger size problems; for the

medium size problems, the level of accuracy only reached 10 -1 for outer it-

erations and 10-2 for inner iteration, winch is not satisfactory. In Problem

1, winch is considered a small sized problem, the two-level Benders performs

better. As vas mentioned previously, we also implemented a single levet Ben-

ders' decomposition where the complicating variables are both Yi and ry2 • Iii

that case ; a single master problem needs to be solved. For Problems 1 to 3,

we reached a preeision of 10 -3 in 17 iterations, which is better than what we

obtained for the two-level version. Our results are consistent with the com-

putational results presented in [50] for the decomposition of the two-stage

stochastic program via Benders' cuts and ACCPM.

5.5 Conclusion

In the present chapter, we formulated the problem of in-house production

and outsourcing planning via the seenario tree for a multi-stage stochastie

program. To address one of the recent challenges facing immufacturers in

ousourcing their products to overseas countries, we considered the stochas-


tic quality and demand in our model formulation. Moreover, we proposed

two types of &compositions namely - the two-level Dantzig-Wolfe reformula-

tion (continu generation) of the dual formulation and the two-level Benders'

&composition of the primai forinulation. The two-level decomposition in

both approaches consists of two master programs and a single subproblem,

where the master programs are solved by ACCPM, and the subproblem is

solved by the simplex method. The two-level column generation via ACCPM

(C:AC-AC) provides a more precise optimal solution compared to that of the

two-level Benders decomposition via ACCPM (B:AC-AC). To improve the

relative precision in solving [TMB], wc changed the C:.AC-AC to C:DW-AC,

where the first level program is solved by the classical Dantzig-Wolfe decom-

position, and the second level is solved by die ACCPM. Among the three

decomposition approaches. the C:DW-AC provides the best level of accuracy

within a small nurnber of iterations.

Chapter

Conclusions

Stochastic programming is mathematical programming obtained by intro-

ducing uncertainty into the problem data and decision process. Such models

are necessary when the effectiveness of current decisions is dependent on fu-

ture events. Fields in which stochastic programming models arise include

economics, financial planning [16], vehicle routing problems [48, 30, 31, 59],

ami telecommunications; many examples can lie fourni in [22, 26] .

Given an appropriate probability space (Q, À. P), this leads to a general

formulation for stochastic programming. Four rnain approaches to stochastic

programming have been developed: expected value solution, the scenario

analysis, the chance constrained pneu:num:lig, and the MC:MUSC formulation.

The recourse formulation or two-stage stochastic programming problem lias

a second stage that is &pendent on the decision vector from the first stage.

This model cari be extended to the multi-stage case when a set of decisions

is to be made over tirne up to horizon T.

The deterministic equivalent to the multi-stage recourse problem can be

Conclusions 138

created by replacing the expectations by probability weighted summations

over the finite number of data paths and writing deterministic constraints for

each path. The size of the associated constraint matrix increases exponen-

tially with the nuinber of time stages and the number of outcomes for cadi

random vector. Each of the data paths is called a scenario.

Multi-stage stochastic programming constitutes a very large-scale opti-

mization problem. Sampling methods are required to reduce the large di-

mensionality of realistic deterministic equivalent problems. In addition, par-

allel programming and decomposition principles are designed for multi-stage

programs to solve the problems more quickly or with more details.

Paraffelization can be donc within the framework of different solution

methods of stochastic programming. For example, parallelization of the in-

terior point method means Cholesky factorization of the matrix, which is

used in calculating the iterative step. Parallelization of the simplex method

on the other hand is extremely difficult. Dantzig et al. [18] have a code for

parallel, sample-based L-Shaped me.thod. A parallel nested &composition

code has been developed by Birge et al. [7].

The large-scale deterministic equivalent to multi-stage recourse problems

cari be solved by the simplex or interior point methods. Alternatively, decom-

position methods such as Benders' &composition or Dantzig-Wolfe decom-

position can be employed. These decomposition principles are translated into

the L-shaped method in stochastic linear programming [72]. The L-shaped

method lias been extended to the nested decomposition method and devel-

oped as a computer code for linear multi-stage recourse problems [29]. The

L-shaped method is viewed as a simplex-based decomposition principle in

Conclusions 139

contrast to the methods which are based on interior point algorithms such

as the Analytic Conter Cutting Plane Method (ACCPM) [33].

The ACCPM combines the interior point algorithm with a cutting plane

method providing a. powerful tool for solving large-scale optimization prob-

lems. The ACCPM was first implemented for solving two-stage stochastic

programming with recourse [3]• This specialized implementation is success-

fully used for practical applications such as telecommunication network prob-

lem [1, 53]. In this thesis we implement a two-level decomposition method via

the ACCPM using column generation technique or Benders' cuts for multi-

stage stochastic programming in the following n'aimer: ACCPM solves the

t'ulster programs and passes forward the current central solution to the sub-

problems, while information passes backward in the form of columns or cuts

to be added to the master programs. In a two-level ACCPM, the analytic

center of the localization provides a central dual price in cadi n'aster program

level to be carried to the subproblems. This provides a better alternative to

simplex dual prices which are used in the standard Dantzig-Wolfe decompo-

sition, and helps the method converge in less iterations.

Our computational results show that the two-level ACCPM using the

column generation technique works much botter than the two-level standard

Dantzig-Wolfe decomposition. In some instances of a multi-stage capacity

planning and technology acquisition problem (sec Chapetr 4), the number of

iterations in the standard Dantzig-Wolfe decomposition is about 300 times

more than the two-level ACCPM. To be able to compare the results of the

two-level ACCPM via Benders' cut against the two-level ACCPM via colurnn

generation, we implemented both approaches 011 a multi-stage stochastic in-

Conclusions 140

bouse production and outsourcing planning problem in Chapter 5. The com-

putational results show that the two-level ACCPM via Bender's cut failed

for the larger size problems whereas for the medium size problems, the re-

suit vas not veiy satisfactœy. To improve the relative precision in solving

the saine problem in Chapter 5, we changed the two-level ACCPM in the

following manner: the first level master program is solved by the classical

Dantzig-Wolfe decomposition while the master program at the second level

is solved by the ACCPM. Since the dimension of the first master program in

multi-stage stochastic in-house production and outsourcing planning prob-

lem is small, this approach provides the best level of accuracy within a simili

number of iterations.

In the future, we would like to efficiently solve multi-stage stochastic mix-

integer programs. To do so, we eau combine our two-level column generation

technique and the branch-and-price method. Branch-and-price is a gener-

alization of LP based branch-and-bound to handle integer progranis that

contain a huge number of variables. In tins general framework, each node of

the search tree is solved via column generation and branching occurs when

no columns price out to enter the basis and the LP solution does not satisfies

the integrality constraints. Note that this approach eau also be combined

with cutting plane algorithms. In this case, it simply suffices to apply the

decomposition procedure on the augmented problem. in [25], it is shown

how to adapt the 1)n:inch-and-price method when ACCPM is used as the LP

solver. Specialized branching and cutting strategies will have to be developed

to solve multi-stage stochastic mix-integer programs in a reasonable ïunount

of computer time.

Bibliography

[1] Andrade 11, Lisser A. Maculan N. and Plateau G. Telecommunication net-

work capacity design for uncertain demand. Computational Optimization

and Applications 2004; 29(2): 127-146.

[2] Babonneau IF, du Merle 0, and \fiai J-P. Solving large-scale linear mul-

ticommodity flow problems with an active set strategy and proximal-

ACCPM. Operations Research 2006; 54(1): 184 -197

[3] Bahn 0, du Merle 0, Coffin J-L, and Vial J-P. A cutting plane method

from anal tic centers for stochastic prograrnming. In: Nondifferentiable

and Large Scale Optimization. Mathematical Programming 1995; B 69:

45-73.

[4] Benders JF. Partitioning procedure for solving mixed-variables program-

ming problems. Numerische Mathematik 1962; 4: 238-252.

[5] Berkelaar A, Gromicho JAS, Kouwenberg R, and Zhang S. A primal-dual

decomposition algorithm for multi-stage stochastic convex programming.

Mathematical Programming 2005; A 104: 153-177.

BIBLIOGRAPHY 142

[6] Birge JR. Decomposition and partitioning methods for multi-stage

stochastic linear programs. J. Operations Research 1985; 33: 989-1007.

[7] Birge JR, Donohue CJ, Hohnes DF, and Svintsiski OG. A parallel impie-

mentation of the nested decomposition algorithm for multi-stage stochastic

linear programs. Tech Report 94-1, Department of Industrial and Opera

tions Engineering, University of Michigan 1994; 327-352.

[8] Birge JR and Holmes DF. Efficient solution of two-stage stochastic linear

programs using interior point methods. Computational Optimization and

Applications 1992; 1: 245-276.

[9] Birge JR and Louveaux FV. A mufti-eut algorithm for two-stage stodras-

tic linear programs. European J. of Operations Research 1988; 384-392.

[10] Birge JR and Louveaux FV. Introduction to stochastic progrannning.

Springer Series in OR 1997; New York.

[11] Birge JR and Wets J-B. Designing approximation scheme for stochastic

optimization problems, in particular for stochastic programs with recourse.

Mathematical Programming study 1986; 27, 54-102.

[12] Canez L, Platts K, and Probert D. Developing a framework for make-or-

buy decisions. International Journal of Operations and Production Man-

agement, 20, 11, 1313 - 30.

[13] Charries A and Cooper W. Response to decision problems under risk and

chance constrained programming dilemmas in the transition. Management

Science 1983; 29, 750-753.

BIBLIOGRAPHY 143

[14] Chen Z, Li S. and Tirupati D. A scenario-based stochastic programming

approach for technology and capacity planning. Computers and Operations

Research 2002; 7: 781-806.

[15] Colommbo M, Gondzio j, and Grothyey A. A warm start for large-

seale stochastic linear programs. Technical Report MS-06-004, School of

Mathematics. University of Edinburgh, 2006.

[16] Consigli G and Dempster MA. Dynamic stochastic programming for

asset liability management. Aimais of Operations Research, 1996.

[17] Dantzig GB and Glynn PW. Parallel processors for planning under un-

certainty. Aimais of Operations Research 1990; 22, 1-21.

[18] Dantzig GB, Ho JK, and Infanger G. Solving stochastic linear programs

on a hypercube multi-computer. Technical Report SOL 91-10. Department

of Operations Research, Stanford University 1991.

[19] Dantzig GB and Wolfe P. The decomposition principle for linear pro-

grains. Operations Research 1960; 8, 101-111.

[20] De Ghellinck G and \fiai J-P. A polynomial Newton method for linear

programming. Algorithmica 1986; 1, 425-453.

[21] Degraeve Z and Jans R. A new Danzig-Wolfe reformulation and brandi-

and-priee algorithin for the capacitated lot sizing problem with set

up times. Technical Report, 2003; Erasmus University Rotterdam, the

N eat her land .

BIBLIOGRAPHY 144

[22] Dempster MA. Introduction to stochastic programming. In: M. A.

H. Dempster (Ed.), Stochastic Programming, Academie Press,1980; New

York.

[23] Dempster MA. On stochastic programming II: Dynamic problems under

risk. Stochastics 1988; 25, 15-42.

[24] Den Hertog D. Interior point approach to linear quadratic and convex

programming. Kluwer Academie Piiblishers 1994; London.

[25] Elhedhli S and Goffin J-L. Efficient production-distribution system de-

sign. Management Science, 2005, 51:7, 1151-1164.

[26] Ermoliev Y and Wets RJ. Numerical techniques for stochastic optimiza-

tion. Springer-Verlag 1988; Berlin.

[27] Fisher ML. The Lagrangian relaxation method for solving integer pro-

gramming problems. Management Science 1981; 27:1, 1-18.

[28] Fragnière E, Gondzio J, and Via.! .1-P. Building a,nd solving large-scale

stochastic programs on an affordable distributed computing system. An-

nais of ()pendions Research 2000; 99,167-187.

[29] Gassman HI. MSLiP: a computer code for the multi-stage stochastic

linear programming problem. Mathematical Programming 1990; 47: 407-

423.

[30] Gendreau M, Laporte G, and Seguin R. An exact algorithm for the

vehicle routing problem with stochastic demand and customers. Trans-

portation Science 1995; 29, 143-155.

BIBLIOGRAPHY 145

[31] Gendreau M, Laporte G, and Seguin R.. Invited review: Stochastic ve-

hicle routing. Europeïm J. Operations Research 1996; 88, 3-12.

[32] Coffin J-L, Gondzio J, Sarkissian R, and Via]. J-P. Solving nonlinear

multicommodity flow problems by analytie center cutting plane method.

Mathematical Programming 1996; 76, 131-154.

[33] Goffin J-L, Haurie A. and Via]. J-P. Decomposition and nondifferentiable

optimization with the projective algorithm. Management Science 1992; 38:

284-302.

[34] Coffin J-L and Vial J-P. Cutting planes and column generation tech-

niques with the projective algorithm. Journal of Optimization Theory and

Applications 1990; 65(3), 409-429.

[3,5] Goffin J-L and \trial J-P. On the computation of weighted analytic centers

and dual ellipsoids with the projective algorithm. Mathematical Program-

ming 1993; 60, 81-92.

[36] Goffin J-L and Via! J-P. Multiple cuts in analytic center cutting plane

method. SIAM Journal of Optimization 1997; 11(1): 266-288.

[37] Gondzio J and Sarkissian R. Colunm generation -with a primai-dual

method. Logilab Technical Report 1997; 96.6.

[38] Gondzio Sarkissian R, and Vial J-P. Using an interior point method

for the master problem in a decomposition approach. European Journal of

Operations Research 1997; 101: 577-587.

BIBLIOGRAPHY 146

[39] Herald Tribune-Business. Mattel recalls millions more Chinese-made

toys. http://www.iht.com/articles/2007/08/14/business/toy.php

[40] Herbig PA and O'Hara BS. The future of original equipment manu-

facturers: a 'natter of partnerships. Journal of Business and industrial

Marketing, 1994; 9, 38 - 43.

[41] Higle JL and Sen S. Stochastic decomposition: An algorithm for two-

stage linear program with recourse. Mathematics of Operations R.esearch

1991; 16, 650-669.

[42] Infanger G. Monte Carlo (importance) sampling within the benders de-

composition algorithm for stocha:stic linear programs. Annal of Operations

Research 1992; 39 69-95.

[43] Kelley JE. The cutting plane method for solving convex programs. Jour-

nal of SIAM 1960; 8, 703-712.

[44] Kibzune AI and Kan YS. Stochastic programming problems with prob-

ability and quintile functions. John Wiley Inc 1996; U.K.

[45] King A and Rackafellar RI. Asymptotic theory for solutions in general-

ized M-estimation and stochastic programing. Mathematics of Operations

Research 1993; 18 148-162.

[46] Lageweg BJ. Lenstra JK, Rinnooy Kan AH, and Stougie L. Stochastic

integer programming. Statistica Neerlandica 1985; 39, 97-113.

BIBLIOGRAPHY 147

[47] Laporte G ami Louveaux FV. The integer L-shaped method for stochas-

tic integer programs with complete recourse. Operations Research Letters

1993; 13, 133-142.

[48] Laporte G, Louveaux FV, and Mercure H. An exact solution for a priori

optimization of the probabilistic traveling salesman problem. Operations

Research 1994; 42, 543-9.

[49] Lee HL and Lee CY. Building supply chain excellence in emerging

economies. Springer US. 2007, V. 98.

[50] Lisser A. Sarkissian R, and Via! J-P. Optimal joint synthesis of base ami

reserve telecommunication networks. Technical Report, Logilab, Depart-

ment of Management Studies, University of Geneva, Switzerland, 1995.

[51] Li S, Loulou R, and Rahman A. Technological progress and technology

acquisition: strategic decision imder uncertainty. Production and Opera-

tions Management 2003; 12(1): 102-119.

[52] Linderoth jT ami Wright SJ. Implementing a &composition algorithm

for stochastic programming on a computational grid. Computational Opti-

mization and Applications, special issue on Stochastic Programming 2003;

24, 207-250.

[53] Lisser A, Ouorou A, Vial J-P, and Gondizo J. Capacity planning under

uncertain demand in teleconnnunication networks. France Telecom Tech-

nical Report, FTRD, France, 1999.

[54] Liibbecke ME and Desrosiers J. Selected TOpieS iii Colunin Generation.

Operations Research 2005; 53(6): 1007-1023.

BIBLIOGRAPHY 148

[55] Martin RK. Large scale linear and integer optimization: a unifie(' ap-

proach. Kluwer Academie Publishers,1999.

[56] Mitchell JE and Todd M. Solving combinatorial opthnization proble.ms

using Karmarkar's algorithm. Mathematical Programming 1992; 56: 245-

284.

[57] Mulvey JM and Ruszczynski A. A new scenario &composition method

for large scale stochastic optimization. Operations R.esearch 1995; 43, 477-

490.

[58] Noël M-C, and Smeers Y. Neste(' decomposition of multi-stage Non-

linear programs with recourse. Mathematical Programming 1987; 37, 131-

152.

[59] Norkin V. Ermoliev Y. and Ruszczynski A. On optimal allocation of

indivisibles under uncertainty. J. Operations Research 1998; 46, 381-395.

[60] Norkin V, Pflug G, and Ruszczynski A. A brandi and bound method

for stochastic global optimization. Mathematical Programming 1998; 83,

425-450.

[61] Oukil A. Ben Amor H. Desrosiers J. and El Gueddari H. Stabilized

column generation for highly degenerate multiple-depot vehicle scheduling

problems. Computers and Operations Research 2005; 34(3): 817-834.

[62] Rasekh L and Desrosiers J. A two-level interior-point &composition

algorithm for inulti-stage stochastic capacity planning and technology ac-

quisition. HEC Montréal, Les Cahiers du GERAD G-2008-09, 18p.

BIBLIOGRAPHY 149

[63] Rockafellar RT and Wets RJ-B. A Lagrangian fuite generation tech-

nique for solving linear-quadratic problems in stochastic programming.

Mathematical Programming 1986; 28: 63-93.

[64] Rockdellar R.T and -Wets R.J-B. Scenario and policy aggregation in op-

timization under uncertainty. Mathematics of Operations Research 1991;

16(1): 119-147.

[65] Ruszczynski A. parallel decomposition of multi-stage stochastic pro-

gramming problems. Mathematical Programming 1993; •8A. 201-228.

[66] Sethi AK and Sethi SP. Flexibilty in manufacturing: a survey. Interna-

fichai Flexible Manufa:cturing Systems 1990; 2: 289-328.

[67] Shapiro A. Asymptotic analysis of stochastic progrïuns. Aimais of Op-

erations Research 1991; 30, 169-186.

[68] Shapiro A. Stochastic programming by monte carlo simulation methods,

published electronically. In: Stochastic Programming E-Print Series, 2000.

[69] Stougie L. Design and analysis of algorithms for stochastic integer pro-

grumning. CWI Tract, centrum voor Wiskunde en Informatica Amester-

dam 1987; 37.

[70] Sun j and Liu X. Scenario formulation of stochastic programs and the

homogeneous self-dual interior point method. To appear in INFORMS

Journal on Computing.

[71] .Vanderbei Rj. Linear programming: foundations and extensions,

Kluwer Academie Publishers, Boston, MA, 1996.

BIBLIOGRAPHY 150

[72] Van Slyke R and .Wets Rj-B. L-shaped linear programs with application

to optimal control and stochastic programming. SIAM J. Appl. Math.

1969; 17: 638-663.

[73] Wallace SW and Yan TC. Bounding multi-stage stochastic programming

from above. Mathematical Programming 1993; 61, 111-129.

[74] Wets R.J. Stochastic programming with fixed recourse: the equivalent

deterministic problem. SIAM Review 1974; 16, 309-339.

[75] Nets Rj. Stochastic programming. In: G.L. Nemhauser, A.H.G. Rin-

nooy Kan, and M.J. Todd (Eds), Handbooks of Operations Research ami

Management Science 1990; 1.

[76] Wittrock RJ. Advances in nested decomposition algorithm for solving

staircase linear programs. Technical Report SOL 83-2, System Optimiza-

tion Laboratory, Stanford University, 1983.

[77] Wright SJ. Primal-dual interior point methods. SIAM, Philadelphia,

1997.

[78] Ye Y. Interior point algorithms, theory and analysis. John Wiley Inc,

1997, U.K.

[79] Zhao G. A log-barrier method with Benders decomposition for solving

two-stage stochastic linear programs. Mathematical Programming 2001;

90: 507-536.

Documents

École des Hautes Études Commerciales ... - École de gestion