DUALITY THEORY FOR COMPOSITE GEOMETRIC PROGRAMMING

Duality Theory for Composite

Geometric Programming

Ya-Ping Wang

Department of Industrial Engineering

University of Pittsburgh

December 14, 2012

DualityTheoryforCompositeGeometricProgramming

Page 1 of 27

Geometric Program (GP)

Primal posynomial program (GP)

(GP) 0inf ( ) . . ( ) 1, : {1,..., }m k

RG s t G k K p

0<tt t

Note that there are

m design variables, 1( , , ) , :{1,..., }mt t J m t 0 ,

each Gk(t) is a sum of terms indexed by the set [k]:

[ ]( ) : ( ), 0,1, , , k ii k

G U k p

t t

Across the p+1 different functions there are a total of

n terms 1, , , {1, , };whereU U I nn

, ( ) : , 0, ija

i i j i ijj Ji I U C t C a R

t ,

These terms are sequentially distributed into the (p+1)

problem functions as follows:

0 1[0] [1] [ ]; , 0< | [ ] |p kI p n n n n n k

The “exponent matrix” is given by ,, 1:[ ]n m

ij i ja A


Page 2 of 27

Equivalent Formulations of (GP)

In the variables = ln , , j jz t j J

ln ( ) ln ln , , lni i ij j i i ij JU C a t c i I c C

it a z

[ ]ˆln ( ) ln exp[ ] : ( ), .i

k i ki kG c g k K

t a z z

A convex formulation of GP:

(GP)z: 0inf ( ) . . ( ) 0, m k

Rg s t g k K

zz z


Page 3 of 27

Equivalent Formulations of (GP) (cont’d)

In the variables =: ix i Iia z,

[ ]( ) ln ( ) ln exp( ) : ( )k k

k k i ii kg G x c geo

z t x c

where [ ]

( ) : ln[ exp ] : knkii k

geo x R R

x is called a

geometric function (also called logexp(x)), [ ][ ]ki i kx x

and [ ][ ]ki i kc c .

A GGP (Generalized Geometric Programming)

formulation of GP: (GP)x

0 0 inf ( ) . . ( ) 0, ,k kgeo s t geo k K x x c x c x P

where | m nR R Az zP is the column space of A.


Page 4 of 27

Dual Posynomial Program (GD)

ˆ [ ]

0

1

[ ]

sup ( ) : (Dual function)

. . 0, , 1 (normality condition)

0, (orthogonality conditions)

where : ,

i

ni k ik K i k

R

i

n

ij ii

k ii k

V C

s t i I

a j J

ˆ .k K

Each dual variable i corresponds to a primal term iU .

Degree of difficulty is defined as ( 1)d n m

The log-dual function

0 [ ]

1 1

( ) : ln ( ) ln /

( ln ) ln

p

i i k ik i k

pn

i i i k ki k

v V C

c

is concave on its domain of definition, and differentiable

in the interior of its domain.


Page 5 of 27

Main Lemma of GP

Lemma: If t is feasible for primal program (GP) and is

feasible for dual program (GD), then

0 ( ) ( )G Vt .

Moreover, under the same conditions 0 ( ) ( )G Vt if, and

only if, one of the following two sets of equivalent

extremality conditions holds:

I. ( ) 1, , (1)

ˆ( ) ( ), [ ], (2)

kk

i k k i

G k K

G U i k k K

t

t t

II. 0( ) ( ), [0] (1)

( ), [ ], (2)i

ik i

U G i

U i k k K

t t

t

in which case t is optimal for primal program (GP) and

is optimal for dual program (GD).


Page 6 of 27

Dual to Primal Conversion at Optimality

When 0 ( ) ( )G Vt ,

we get from condition II on p.5 the equations

( ), [0]

( ), [ ], , and 0

ii

i k k

V iU

i k k K

t

,

which are log-linear:

1

ln( ( )), [0]

ln , [ ], for which 0

mi

ij j ij i k k

V ia z c

i k k K

So, knowing the optimal dual solution an optimal

primal solution can in general be easily recovered by

solving this log-linear system for z and hence for t.


Page 7 of 27

EXAMPLE:

(Maximum Likelihood Estimator of a Bernoulli Parameter)

Suppose that n independent trials, each of which is a

success with probability p, are performed. What is the

maximum likelihood estimator (MLE) of p?

The likelihood function to be maximized is

1, 1( , | ) (1 ) , with , 0 or 1,n n

ns n sn n i if x x p p p s x x i

Example: n=3, m=2: 1

( , ) 0inf s.t. 1n ns s n

p qf p q p q

The exponent matrix and a unique solution from

extremality condition II (2) on p.5: * *

1

1 **2

2 *1

3

=

1

ˆ 1 0 , so

0 1

n n

n nn

n

p q n

s s ns S

s p p Xn n

n s


Page 8 of 27

Prior Extensions to GP

Signomial GP which allow some <0iC non-

convex programs: duality results are not strong

Peterson’s Generalized GP (GGP)More general separable convex programs

Extensions to GP investigated in this thesis

Composite GP (CGP): itself a special case of Peterson’s GGP. It includes as special cases:

Exponential GP (EGP)

Quadratic GP (QGP)

(lpGP)

We start with Exponential GP (EGP) using a motivating example…


Page 9 of 27

A Motivational Example

(Maximum Likelihood Estimator of a Poisson Parameter)

Suppose 1, , nX X are independent Poisson random

variables each having mean λ. Determine the MLE of λ.

The likelihood function to be maximized is

1, 1 1( , | ) / ( ! !), where :n

ns nn n n ii

f x x e x x s x

Equivalently, one can 0 min ns ne . Although this is

not a posynomial, it can be solved as an EGP:

11 1

1

From condition (e) on p.14, we get

1 ˆ / = , so /1

nn n

n

y sy s n S n X

s

This same method also works for Exponential and Normal

parameters…


Page 10 of 27

Exponential Geometric Program (EGP)

In the previous example:

a posynomial term is multiplied by an exponential

factor of another posynomial term.

This gives rise to an EGP problem where some

posynomial term ( )iU t is multiplied by an exponential

factor of another posynomial ( ) : ( )i ll iE V

t t .

Primal EGP problem (EGP): (cf (GP) on p.1)

0inf ( ) . . ( ) 1, ,m kRG s t G k K

0

<tt t

where

[ ]

1

ˆ( ) : ( ) exp ( ) , : {0}

( ) , ,and ( ) ,

{1, , } 1 ,| | 0,

ij lj

k i li k l i

a b

i i j l l jj J j J

i n

G U V k K K

U C t i I V D t l L

L r n i r r r r

t t t

t t

[ ] :ljB b r m


Page 11 of 27

Equivalent formulations of (EGP)

In the variables

A convex form of (EGP): (EGP)z

0inf ( ) . . ( ) 0, m k

Rg s t g k K

zz z , where

[ ]ˆ( ) : ln ( ) ln exp exp , i

k k i li k l ig G c d k K

lz t a z b z +

In the variables : , : , i lx i I l L i la z, b z,

[ ] [ ]( ) ( , ) with exp,k k k kk lg geo h l L z x c ξ d

where

[ ]

[ ]( , ) : ln exp exp( )k k

i li k l igeo x

x ξ

:k

kn rR R R is called an exponential geometric

function, [ ][ ] [ ]

[ ] , [ ] , :irk i i ki k l l i ii k

R r r ξ ξ ξ ,

and [ ]kd is similarly defined.

= ln , , j jz t j J


Page 12 of 27

Equivalent formulations of (EGP) (cont’d)

A GGP formulation of (EGP):

(EGP)x,ξ

0 0 [0] [0]

( )

[ ] [ ]

inf ( , )

. .

( , ) 0, ,( )

n rR R

k k k k

geo

s t

geo k K

x,x c ξ d

x c ξ d x,ξ

P

where | m n rM R R z zP is the column space of the

composite exponent matrix A

BM

.


Page 13 of 27

Dual of EGP program (EGD) (cf (GD) on p.4)

ˆ( ) ( , ) [ ]

0

1 1

sup ( , ) :

. . 0, , 0, , 1,

0, (Orthogonality Conditions)

0

i l

n r

y

i k l i

R R i k l ik K i Ii l

i l

n r

ij i lj li l

i

C D yV e

y

s t y i I l L

a y b j J

y

,

yy

[ ]

0, , (*) p.16

ˆ where : , , : =

l

k i li k l L

l i i I

y k K

Note that ( ), ( )i i l ly U V t t

and degree of difficulty ( 1)d n r m →p.4

The log-dual function (cf p.4)

ˆ [ ]

( , ) ln / ln /i i k i l l i li k l ik K i I

v y C y D y

y

is usc proper concave in( , )y , and differentiable in the

interior of its domain.


Page 14 of 27

Main Lemma of EGP

Lemma If t is feasible for primal program (EGP) and

( , )y is feasible for dual program (EGD), then 0 ( ) ( , )G V y t

Moreover, under the same conditions, 0 ( ) ( , )G V y t if

and only if one of the following two sets of equivalent

extremality conditions hold: (cfI&IIonp.5)

I ' Condition (e) & I ( ) 1, ,

ˆ( ) ( ), [ ],

kk

i k k i

G k K

y G U i k k K

t

t t

II ' Condition (e) & II 0( ) ( ), [0]

( ), [ ],

ii

k i

U G iy

U i k k K

t t

t

where (e) ( ), , l i lyV l i i I t

in which case t is optimal for primal program (EGP) and

( , )y is optimal for dual program (EGD).

p.9 Solve the previous motivational Example.


Page 15 of 27

Composite Geometric Program (CGP)

Primal CGP Program (CGP) (cf (EGP) on p.10)

0inf ( ) . . ( ) 1, m kRG s t G k K

0

<tt t , where

[ ]ˆ ( ) ( ) exp (ln ( )) ,k i l li k l i

G U h V k K

t t t ,

:lh R R is a differentiable and strictly convex function.

Note that:

(QGP): if 212( ) ,lh l L

(lpGP): if ( ) | | / , where 1,lpl l lh p p l L


Page 16 of 27

Composite Geometric Program (CGP) (cont’d)

A Convex Form of (CGP): (CGP)z

0 inf ( ) . . ( ) 0, m kRg s t g k K

zz z , where

[ ]( ) : ln ( ) ln exp i l

k k i l li k l ig G c h d

z t a z b z

A GGP Form of (CGP): (CGP)x,

0 [0] [ ]0

( )inf ( , ) . . ( , ) 0, , ( )

n r

k kk

R Rf s t f k K

ξ ξ ξ

x,x x x, P

with [ ]( , ) ( , ) : ln expk i l li k l i

f geo x h


Page 17 of 27

Dual program (CGD) (cf (EGD) on p.13)

*

ˆ( ) [ ]

0

1 1

sup ( , ) : exp ( / )

. . 0, , 1, (Normality Condition)

0, (Ortho

i

n r

y

i kl l i l l i

R R l ii kk K i Ii

i

n r

ij i lj li l

CV d y h y

y

s t y i I

a y b j J

y,η

y

[ ]

gonality Conditions)

, , (**)

ˆwhere : , .

l i l

k ii k

y J l i i I

y k K

where *

lh , the conjugate of lh , has domain interval lJ .

(**) 0l in (EGD) on p. 13 and that in any (CGD)

(*) 0 0, , i ly l i i I

The log-dual function (cf p.13)

*

ˆ [ ]

( , ) ln / ( / )i i k i l l i l l ii k l ik K i I

v y C y d y h y

y

is usc proper concave in( , )y , and differentiable in the

interior of its domain.


Page 18 of 27

Main Lemma of CGP

Lemma: If t is feasible for primal program (CGP) and

( , )y is feasible for dual program (CGD), then

0 ( ) ( , )G V t y .

Moreover, under the same conditions, 0 ( ) ( , )G V t y if

and only if one of the following two sets of equivalent

extremality conditions hold: (cfI’&II’onp.14)

I '' Condition (c) & I ( ) 1, ,

ˆ( ) ( ), [ ],

kk

i k k i

G k K

y G U i k k K

t

t t

II '' Condition (c) & II 0( ) ( ), [0]

( ), [ ],

ii

k i

U G iy

U i k k K

t t

t

where (c) ' (ln ( )), , ll i ly h V l i i I t

in which case t is optimal for primal program (CGP) and

( , )y is optimal for dual program (CGD).

(lpGP): (c) 1| ln ( ) | sgn(ln ( )), ,lpl i l ly V V l i i I t t


Page 19 of 27

First Duality Theorem of CGP

NOTE: The statements for EGP, QGP, lpGP cases are

almost the same.

Theorem: Suppose that primal program (CGP) is super-

consistent. Then the following three conditions are

equivalent:

1) t’ is a minimal solution to (CGP).

2) There exists a vector ' pR for z′ (where z′= ln t′)

such that ( ', ')z forms a saddle point of ( , )l z .

where ( , )l z is the Lagrangian of (CGP)z.


Page 20 of 27

3) There exists a vector ' pR for z′ (where z′= ln t′)

such that ( ', ')z satisfies the KKT conditions for

(CGP)z:

in which case the set of all such vectors 'λ is a non-empty

compact convex subset of pR , and the dual program

(CGD) also has a maximum solution ( ', ')y such that

0min(CGP) ( ') ( ', ') max(CGD)G V t y

and they satisfy the extremality conditions I ''& II '' on

p.18.

(Perfect Duality)

' '

' '0ˆ

( ) 0, ( ) 0, ( ) 0,

( ) ( , ') ( ) , 1

k k k k

z k kk K

a g g k K

b l g where

0

z' z'

z' z'


Page 21 of 27

Second Duality Theorem of EGP

From linear programming duality theory:

Of the following two linear systems, exactly one has a

solution (where :A

BM n r m

):

(I) Find z with 0 0

A

Bz

(II) Find >0 with T TA B

y

y 0

We say that program (EGD) is canonical if system (II)

has a solution when M is the program’s composite

exponent matrix.

Theorem: Suppose that primal program (EGP) is

consistent. Then the minimum set of program (EGP)z is

non-empty and bounded if, and only if, dual program

(EGD) is canonical, in which case program (EGP) has a

minimum solution t′.


Page 22 of 27

Second Duality Theorem of lpGP*

* This theorem also applies to its special case QGP.

Again, from linear programming duality theory

Of the following two linear systems exactly one has a

solution (where :A

BM n r m

):

(I) Find z with , 0 Az 0 Bz 0

(II) Find with , and T TA B

y

y 0 y 0

We say that program (lpGD) is canonical if system (II)

has a solution when M is the program’s composite

exponent matrix.

Theorem 7.3.2 Suppose that primal program (lpGP) is

consistent. Then the minimum set of program (lpGP)z is

non-empty and bounded if and only if its dual program

(lpGD) is canonical.


Page 23 of 27

SUMMARY OF CONTRIBUTIONS

Extension of traditional GP models to

o to the more general Exponential GP models,

o to the even more general Composite GP models,

which include as important special cases:

(EGP)

(lpGP)

(QGP)

Showing that all of these are special cases of

Peterson’s GGP models (for which he has given a

Main Lemma but not a First or a Second duality

theorem).


Page 24 of 27

CONTRIBUTIONS (cont’d)

For each extension:

o multiple direct proofs of the Main Lemma, with a

2nd set of equivalent Extremality Conditions.

For the First Duality Theorem, proof that

o a superconsistent primal program (CGP) has a

minimal solution 't if, and only if, there exists a

vector ' pR such that ( ', ')z forms a saddle

point of the Lagrangian ( , )l z and if, and only

if, it satisfies the KKT conditions for (CGP)z,

o in which case the set of all such Lagrange

multiplier vectors λ is a non-empty compact

convex subset of pR , whereas the original

theorem is a “If…then” statement and only

showed the existence of λ for the (GP) case only.


Page 25 of 27

For the Second Duality Theorem, proof that

o for the special (and more important) cases of

(QGP), (lpGP) and (EGP),

the minimum set of a consistent primal

program is non-empty and bounded if and

only if its dual program is canonical

the original theorem showed the existence of

a primal solution for the (GP) case only,

when its dual program is canonical.

Sensitivity Analysis: If *z is optimal for (CGP)z:

* * ** * *0 0 0

* ** * * *0 0

( ) ( ) ( ), , ,

( ) ( ), , where ln

i l ki l k

i j l j k kij lj

g g gy

c d b

g gy z z b B

a b

z z z

z z

Dual to primal conversion of optimal solutions

when there is no duality gap: 0 ( ) ( , )G V y t .


Page 26 of 27

Directions for Future Research

1. Development of proofs of a strong version of the

First Duality Theorem of (CGP): If a primal program

(CGP) is superconsistent and has a finite infimum,

then the dual program (CGD) has a maximum

solution ( ', ')y such that

inf(CGP) max(CGD) ( ', ')V y

2. Development of computational algorithms.


Page 27 of 27

QUESTIONS