OR-1 2011 1
Chapter 5. The Duality Theorem
Given an LP, we define another LP derived from the same data, but with different structure. This LP is called the dual problem ( 쌍대문제 ).
The main purpose to consider dual is to obtain an upper bound (estimate) on the optimal objective value of the given LP without solving it to optimality. Also dual problem provides optimality conditions of a solution x* for a LP and help to understand the behavior of the simplex method.
Very important concept to understand the properties of the LP and the simplex method.
OR-1 2011 2
Approach to find optimal value of an opt. problem (max form):
Find lower bound and upper bound so that for optimal value , we have
Lower bound : usually obtained from a feasible solution.
If x* is a feasible solution to LP, c’x* provides a lower bound.Upper bound : usually obtained by solving a relaxation of the problem or by finding a feasible solution to a dual problem.
ex) linear programming relaxation of an integer program:
Given IP, max c’x, Ax b, x 0 and integer, consider the linear programming relaxation, max c’x, Ax b, x 0.
Let z* be the optimal value of IP and z’ be the optimal value of LP relaxation. Then z* z’. ( Let x* be the optimal solution of IP. Then x* is a feasible solution of LP relaxation. So there may exist a feasible solution to LP which provides better objective value)
z z*z zzz *
OR-1 2011 3
(continued)
Dual of the LP problem : Let y be a feasible solution to the dual problem. Then the dual objective value provided by y gives an upper bound on the optimal LP objective value. (study in Chapter 5.)
If the lower and upper bound are the same, we know that z* is optimal value. We may need to find the optimal solution additionally, but the optimal value is found.
Although the lower and upper bound may not be the same, from the gap (= upper bound – lower bound), we can estimate the quality of the solution we have.
OR-1 2011 4
Taking nonnegative linear combination of inequality constraints: Consider two constraints and …. (1’)
In vector notation :
If we multiply scalars y1 0 to the 1st constraint and y2 0 to the 2nd
constraint and add the l.h.s. and r.h.s. respectively,
we get ….. (2’)
In vector notation,
Any vector that satisfies (1’) also satisfies (2’), but converse is not true.
Moreover, the coefficient vector in l.h.s. of (2’) is obtained by taking the nonnegative linear combination of the coefficient vectors in (1’)
32 21 xx 42 21 xx
1
2,
2
1,4',3' 2121 aaxaxa where
)43()2()2( 21221121 yyxyyxyy )43()''( 21
22
11 yyxayay
OR-1 2011 5
x1
x2
(0, 0)
x1+2x23
2x1+x24y1(1, 2)+y2(2, 1), y1, y2 0
(1, 2)
(2, 1)
(5/3, 2/3)(y1a1’+y2a2’)x (3y1+4y2)
OR-1 2011 6
ex)
Lower bound : consider feasible solution (0, 0, 1, 0) z* 5
(3, 0, 2, 0) z* 22
Upper bound : consider inequality obtained by multiplying 0 to the 1st , 1 to the 2nd, and 1 to the 3rd constraints and add the l.h.s. and r.h.s. respectively
…… (1)
3532
55835
13..
354max
4321
4321
4321
4321
xxxx
xxxx
xxxxts
xxxx
0,,, 4321 xxxx
583634 4321 xxxx
OR-1 2011 7
Since we multiplied nonnegative numbers, any vector that satisfies the three constraints (including feasible solutions to the LP) also satisfies (1). Hence any feasible solution to LP, which satisfies the three constraints and nonnegativity, also satisfies (1).
Note that all the points satisfying 4x1 + 3x2 + 6x3 + 3x4 58 do not
necessarily satisfy the three constraints in LP.
Further, any feasible solution to the LP must satisfy
( 58)
since any feasible solution must satisfy nonnegativity constraints on the variables and the coefficients in the second expression is greater than or equal to the corresponding coefficients in the first expression.
So 58 is an upper bound on the optimal value of the LP.
(1) ....... 583634 4321 xxxx
3634354 43214321 xxxxxxxx
OR-1 2011 8
Now, we may use nonnegative weights yi for each constraint.
In vector notation,
Objective function of the LP is
321
4321332123211321
355
)583()33()2()5(
yyy
xyyyxyyyxyyyxyyy
)355(532,1(831,5(3,11,1({ 321321 yyyxyyy )}' , , ) , , ) ,
4321 354 xxxx
3532
55835
13..
354max
4321
4321
4321
4321
xxxx
xxxx
xxxxts
xxxx
0,,, 4321 xxxx
OR-1 2011 9
Hence as long as the nonnegative weights yi satisfy
we can use as an upper bound on optimal value. To find more accurate upper bound (smallest upper bound), we want
to solve
Dual problem obtained. Note that the objective value of any feasible solution to the dual problem provides an upper bound on the optimal value of the given LP.
3583,533
12,45
321321
321321
yyyyyy
yyyyyy
321 355 yyy
3583
533
12
45..
355min
321
321
321
321
321
yyy
yyy
yyy
yyyts
yyy
0,, 321 yyy
3532
55835
13..
354max
4321
4321
4321
4321
xxxx
xxxx
xxxxts
xxxx
0,,, 4321 xxxx
OR-1 2011 10
General form
P: primal problem, D: dual problem
maximize
n
jjj xc
1
n
jijij mibxa
1,...,1,subject to
njx j ,...,2,1,0
0
subject to
' maximize
x
bAx
xc
minimize
m
iii yb
1
m
ijiij njcya
1,...,1,subject to
miyi ,...,2,1,0
0
'' subject to
' minimize
y
cAy
yb
(P)
(D)
OR-1 2011 11
Thm: (Weak duality relation)
Suppose (x1, …, xn) is a feasible solution to the primal problem (P) and
(y1, …, ym) is a feasible solution to the dual problem,
then
pf)
Cor: If we can find a feasible x* to (P) and a feasible y* to (D) such that , then x* is an optimal solution to (P) and y* is an optimal solution to (D).
pf) For all feasible solution x to (P), we have
Similarly, for all feasible y to (D), we have
nj
mi iijj ybxc1 1
nj
mi ii
mi
nj ijij
nj
mi jiijjj ybyxaxyaxc1 11 11 1 )()(
nj
mi iijj ybxc1 1
**
nj
mi
nj jjiijj xcybxc1 1 1
**
mi
nj
mi iijjii ybxcyb1 1 1
**
OR-1 2011 12
[Strong Duality Theorem] If (P) has an optimal solution (x1*, …, xn
* ),
then (D) also has an optimal solution, say (y1*, …, ym
* ), and
(i.e., no duality gap, dual optimal value – primal optimal value = 0)
Note that strong duality theorem says that if (P) has an optimal solution, the dual (D) is neither unbounded nor infeasible, but always has an optimal solution.
nj
mi iijj ybxc1 1
**
OR-1 2011 13
Idea of proof: Read the optimal solution of the dual problem from the coefficients of the slack variables in the z-row from the optimal dictionary (tableau).
ex)
3532
55835
13
0354
74321
64321
54321
4321
xxxxx
xxxxx
xxxxx
xxxxz
Note that the dual variables y1, y2, y3 matches naturally with slack variables x5, x6, x7. For example x5 is slack variable for the first constraint and y1 is dual variable for the first constraint, and so on.
OR-1 2011 14
At optimality, the tableau looks
1112195
52
143542
296112
76531
75431
75321
7531
xxxxx
xxxxx
xxxxx
xxxxz
In the z-row of the tableau, the coefficients of the slack variables are –11 for x5, 0 for x6, -6 for x7.
Assigning these values with reversed signs to the corresponding dual variables, we obtain desired optimal solution of the dual:
y1 = 11, y2 = 0, y3 = 6.
OR-1 2011 15
Idea of the proof:
Note that the coefficients of x5, x6, and x7 in the z-row show what
numbers we multiplied to the corresponding equation and add them to the z-row in the elementary row operations (net effect of many row operations)
ex)
Suppose we performed row operations
(row2) 2*(row 1) +(row 2), then ( z-row) 3*(row 2) + (z-row).
The net effect in z-row is adding 6*(row 1) + 3*(row 2) to the z-row and the scalar we multiplied can be read from the coefficients of x5
and x6 in the z-row.
3532
55835
13
0354
74321
64321
54321
4321
xxxxx
xxxxx
xxxxx
xxxxz
OR-1 2011 16
3532
5721417
13
0354
74321
654321
54321
4321
xxxxx
xxxxxx
xxxxx
xxxxz
(ex-continued)
3532
55835
13
0354
74321
64321
54321
4321
xxxxx
xxxxx
xxxxx
xxxxz
(row 1)2 +(row 2)
OR-1 2011 17
3532
5721417
13
17136458225
74321
654321
54321
654321
xxxxx
xxxxxx
xxxxx
xxxxxxz
3532
5721417
13
0354
74321
654321
54321
4321
xxxxx
xxxxxx
xxxxx
xxxxz
(ex-continued)
(row 2) 3 +z-row
OR-1 2011 18
Example – Initial tableau
3532
55835
13
0354
74321
64321
54321
4321
xxxxx
xxxxx
xxxxx
xxxxz
Optimal tableau
1112195
52
143542
296112
76531
75431
75321
7531
xxxxx
xxxxx
xxxxx
xxxxz
OR-1 2011 19
Let yi be the scalar we multiplied to the i-th row and add to the z-row in
the net effect.
Then the coefficient of slack variables in the z-row represent the yi
values we multiplied to the i-th row for i = 1, … ,m.
Also the coefficients of structural variables in the z-row are given as
Now in the optimal tableau, all the coefficients in the z-row are 0, which implies
If we take – yi as a dual solution, they are dual feasible.
njayc mi ijij ,...,1,1
) variablesstructural(for ,...,1,01 njayc mi ijij
ables)slack vari(for ,...,1,0 miyi
njcaymi jiji ,...,1,)(1
miyi ,...,1,0)(
OR-1 2011 20
Also the constant term in the z-row gives the value
So it is the negative of the dual objective value of the dual solution (-yi ), i = 1, ,,, , m
Note that the constant term in the z-row also gives the negative of the objective value of the current primal feasible solution.
So we have found a feasible dual solution which gives the same dual objective value as the current primal feasible solution.
From previous Corollary, the dual solution and the primal solution are optimal to the dual and the primal problem, respectively.
It is the idea of the proof.
mi iiby1
OR-1 2011 21
pf of strong duality theorem)
Suppose we introduce slack variables
and solve the LP by simplex and obtain optimal dictionary with
Let
We claim that yi*, i = 1, … , m is an optimal dual solution, i.e. it
satisfies dual constraints and
mnk
nj jjkkk xczkcxczz 1 1
*** ,0,
micy ini ,...,1,*
mi ii zyb1
**
nj
mnnj jj
nj jjjj xcxczzxcz 1 11
* and Equate
)...,,1(1 mixabx nj jijiin
This equation must be satisfied by any x that satisfies the dictionary (excluding the nonnegativity constraints) since the set of feasible solutions does not change for any dictionary.
nj
nj
mnnj jjjjjj xcxczxc1 1 1
*
OR-1 2011 22
since any feasible solution to dictionary should satisfy this.
above into Substitute mixabx nj jijiin ,...,1,1
nj
nj
mi
nj jijiijjjj xabyxczxc1 1 1 1
** )(
nj
mi
nj
mi jiijjiijj xyacybzxc1 1 1 1
*** )(
nj
nj
mnnj jjjjjj xcxczxc1 1 1
*Have
Now this equation should hold for all feasible solutions to the dictionary. From the initial dictionary, we know that any feasible solution to the dictionary can be obtained by assigning arbitrary values to x1, … , xn and setting
xn+i = bi - j=1n aijxj , ( i = 1, … , m).
Use these solutions. Note that, in the above equation, the variables xn+i do not appear. So it must hold for any choices of xj , j = 1, … , n.
OR-1 2011 23
Equality must hold for all choices of x1, …, xn.
Hence
Hence y* is dual feasible.
Also we have that
Since
yi*, i = 1, … , m is an optimal dual solution and
mi iij
mi jjii njyaccybz 1
*1
** ,...,1,,
mi jiij
mi iijjk cyayackc 1
*1
* 00 n1,...,j,
miyy *ii ,...,1,00*
mi
nj jjii xczyb1 1
***
mi ii
mi
nj jjii yybxcyb 11 1
** duality) (weak feasible dual all for
mi
nj jjii xcyb1 1
**
nj
mi
nj
mi jiijjiijj xyacybzxc1 1 1 1
*** )(