Pinter89 Probabilistic Inequalites

8/10/2019 Pinter89 Probabilistic Inequalites

1/21

Z O R M e t h o d s a n d M o d e ls o f O p e r a ti o n s R e s e a rc h ( 1 9 8 9 ) 3 3 : 2 1 9 - 2 3 9

D eterm inist ic pprox ima tions of Prob abi l ity Inequa l it ies

B y J . P i n t 6 r 1

A b s t r a c t : A s i m p le g e n e r a l f ra m e w o r k f o r d er i v in g exp l i c i t d e t e r m i n i s t i c a p p r o x i m a t i o n s o f p r o b a -

b i l i t y i n e q u a l i ti e s o f t h e f o r m P ( ~ / > a ) ~< ~ i s p r e s e n t e d . T h e s e a p p r o x i m a t i o n s a r e b a s e d o n l i m i t e d

p a r a m e t r i c i n f o r m a t i o n a b o u t t h e i n v o l v e d r a n d o m v a r ia b l es ( su c h a s th e i r m e a n , v a r i a n c e , r a n g e o r

u p p e r b o u n d v a l u e s ). F i r s t t h e c a s e o f a s i n g le r a n d o m v a r i a b l e ~ i s a n a l y s e d , f o l l o w e d b y t h e c a s e s

n

o f i n d e p e n d e n t a n d d e p e n d e n t s u m m a n d s f = ~ ~ i. A s e x a m p l e s o f p o ss i b le a p p l ic a t i o n s , a st o c h a s -

1

t i c e x t e n s i o n o f t h e " k n a p s a c k p r o b l e m " a n d t h e s t o c h a s t i c l i n e a r p r o g r a m m i n g p r o b l e m w i t h

s e p a r a te c h a n c e - c o n s t r a i n t s a r e i n v e s ti g a t e d : w e p r o v i d e a p p r o x i m a t e d e t e r m i n i s t i e su r r o g a te s f o r

t h e s e p r o b l e m s .

Z u s a m m e n f a s s u n g : E s w i r d e i n R a h m e n z u r A b l e i t u n g e x p l i z it e r d e t e r m i n i s t i s c h e r A p p r o x i m a t i o n

f ii r W a h r s c h e i n l i c h k e i t su n g l e i c h u n g e n d e r F o r m P ( ~ ~ a ) ~< a a n g e g e b e n . D i es e A p p r o x i m a t i o n e n

b a s i e r en a u f b e g r e n z t e r p a r a m e t r i s c h e r I n f o r m a t i o n t ib e r d ie b e t e i l ig t e n Z u f a l ls v a r ia b l e n ( w ie i h r

E r w a r t u n g s w e r t , V a r i a n z , W e r t e b e r e ic h o d e r o b e r e S c h r a n k e n ) . Z u e r s t w i r d d e r F a i l e i n e r Z u f a l ls -

n

v a ri a b le n f a n a l ys i er t, s o d a n n w e r d e n S u m m e n v o n u n a b h g n g i g e n S u m m a n d e n ~ = Y, ~ i b e t r a c h t e t .

i=1

A l s B e i s pi e le f ti r m 6 g l ic h e A n w e n d u n g e n w i r d e i n e s to c h a s t i s c h e E r w e i t e r u n g d e s R u c k s a c k -

p r o b l e m s u n t e r s u c h t s o w i e s t o c h a s t i s c h e l i n e a r e P r o g r a m m e m i t s e p a r a b l e n W a h r s c h e i n l i c h k e i t s -

r e s t r i k t i o n e n . F i i r d i e s e P r o b l e m e w e r d e n n / i h e r u n g s w e i s e d e t e r m i n i s t i s c h e E r s a t z p r o b l e m e a n g e -

g e b e n .

K e y w o r d s c h a n c e - c o n s t r a i n t s ; p r o b a b i l i t y i n e q u a l i t i e s; e x p l i c it d e t e r m i n i s t i c a p p r o x i m a t i o n s .

1 J ~t no s P i n t 6 r , R e s e a r c h C e n t e r f o r W a t e r R e s o u r c e s D e v e l o p m e n t ( V I T U K I ) , 1 4 5 3 B u d a p e s t ,

P . O . B o x 2 7 , H u n g a r y .

0 3 4 0 9 4 2 2 / 8 9 / 4 / 2 1 9 2 3 9 $ 2 . 5 0 9 1 9 8 9 P h y s ic a -V e r la g , H e i d e l b e r g


2/21

220 J. Pint6r

1 I n t r o d u c t i o n

L e t ~ = ~ (w) b e a r an d o m v a ri ab le ( r .v . ) w i th c o n t i n u o u s , s t r i c t ly m o n o to n o u s p ro b a -

b i l i ty d i s t r ibu t ion func t ion (p .d . f . ) . We sha l l p resen t a s imple genera l approach to ap -

p ro x im a te p ro b ah i l i t y i n eq u al it ie s o f t h e fo rm

P ~ a ) ~ a (1 .1 )

where --oo< a < ~ , 0 ~< a ~< 1 are g iven values. The ap pro xim atio ns are at ta ine d via

subs t i tu t ing (1 .1 ) by (genera l ly speak ing , s t r i c te r ) de te rm in is t i c inequal i ti es which con-

t a in p a ram e te r s t h a t

partially

charac te r ize the p .d . f , o f ~ (such as i t s mean , var iance ,

range e tc . ) . To mot iva te th i s s tudy , l e t us men t ion tha t e .g . in s tochas t ic (chance-

cons t ra ined) p rogramming , r i sk ana lys i s , re l i ab i l i ty theory e tc . one o f ten fo rmula tes

cer ta in s ta t i s t i ca l requ i rements as p robab i l i s t i c cons t ra in t s which can be b rough t to

fo rm (1 .1 ) , see e .g . Berger (1985 ) , D emp s ter (1980) , Wets (1983) . F req uen t ly , the r .v .

depends on severa l dec i s ion var iab les p lus a number o f (uncon t ro l lab le , unknown)

ran d o m fac to r s i n a co m p l i ca t ed , an a ly t ica l l y n o n - tr ac t ab l e ( im p l i c i t) m an n e r : co n s e -

q u en t l y , ev en n o t ev e ry accu ra t e , b u t

explicit

d e t e rm in i s t i c ap p ro x im a t io n s o f (1 . 1 )

m a y b e o f p rac t i ca l v a lu e .

There ex is t s a fa i r ly ex tens ive l i t e ra tu re conc ern ing d i f fe ren t p robab i l i s t i c inequali -

t ies which - w i th respec t ive spec i f ica t ions o f the i r r .v . 's - y ie ld sharp boun ds , see fo r

ex am p le B ah ad u r an d R ao (1 9 6 0 ) , B en n e t t (1 9 6 2 ) , C h e rn o f f (1 9 5 2 ) , Daws o n an d

San k o f f (1 9 6 7 ) , Du p aco v a (1 9 8 0 , 1 9 8 7 ) , Ga l am b o s (1 9 7 7 ) , G o d win (1 9 5 5 ) , Ho e f fd in g

(1963 ) , Huang , Z iem ba and Ben-Tal (1977) , K al l and S toya n (1982 ) , Kankov a (1977) ,

Kle in Hanev .e ld (1985) , Kwere l (1975) , Madansky (1960) , M6ri and Sz6kely (1985) ,

Ok am o to (1 9 5 8 ) , Pe rcu s an d Pe rcu s (1 9 8 5 ) , P l a t z (1 9 8 5 ) , P in t6 r (1 9 8 5 ) , P ro h o ro v

(1959) , Sa the , Pradhan and Shah (1980) , Seppala (1975) , S inha (1963) , Sz~n ta i

(19 85) and Yudin (1980) . In th i s pape r - based on severa l c i t ed re ference s - m ean /

range , mean /var iance and me an /var iance / ran ge app rox im at ions w i ll be g iven . The un i -

var ia te case is fo l lo we d b y the mo re genera l case o f sums o f r .v . 's . Tw o possib le appl i-

cat ions serve as i llus trat ive exam ples: a s toch ast ic vers ion o f the so-cal led kna psa ck

p ro b l em o f i n t eg e r p ro g ram m in g an d t h e s t o ch ast i c l in ea r p ro g ram m in g p ro b l em w i th

separa te chance -cons t ra in t s .

2 T h e C a s e o f a S i ng le R a n d o m V a r i a b l e

Cons ider the p rob ab i l i ty inequ al i ty (1 .1 ) w i th g iven para m eter va lues a and a : the ob-

serva tions be low p erm i t the der iva t ion o f

different deterministic surrogates

fo r (1 .1 ) .


3/21

Determ inistic Ap proxim ations o f Probab ility Inequalities 221

L e m m a 2 .1 : S u p p o s e t h a t a p r o b a b i l i s t i c r e l a t i o n

P ( ~ > I f ( c , ~ ) ) < , g ( c , ~ )

( 2 . 1 )

c a n b e g i v e n f o r t h e r . v . ~ ( a n d a l l v a l u e s o f 1 3), w h e r e 13 s y m b o l i z e s s o m e p a r a m e t e r o f

t h e i n e q u a l i t y ( 2 . 1 ) , c = c~ r e p r e s e n t s q u a n t i t a t i v e i n f o r m a t i o n a b o u t ~ ( s u c h a s i ts

m e a n , v a r i an c e , h ig h e r m o m e n t s , r a n g e e t c .) , f a n d g ar e k n o w n c o n t i n u o u s f u n c t i o n s

o f th e i r a rg u m e n t s . I f - f o r s o m e ad m i s si b le p a r a m e t e r i z a t io n s - t h e e q u a t i o n g( c, [3) = a

h a s a s o l u t i o n i n / 3 :

= k ( c , a ) ( 2 . 2 )

t h e n t h e e x p l i c i t d e t e r m i n i s t i c r e l a t i o n

f c , k c , ~ ) ) < ~ a 2 .3 )

i m p l i e s t h e p r o b a b i l i t y i n e q u a l i t y ( 1 . 1 ) .

L e m m a 2 .2 : A s s u m e a g ai n t h a t t h e p r o b a b i l i t y i n e q u a l i t y ( 2 . 1 ) h o ld s a n d s u p p o s e t h a t

f o r s o m e p a r a m e t r i z a t i o n s t h e e q u a t i o n f ( c , 13) = a h a s a s o l u t i o n i n 1 3 :

13 = h( c, a) ( 2 . 4 )

T h e n t h e e x p l i c i t d e t e r m i n i s t i c r e l a t i o n

g c , h c , a ) ) < ,~ 2 . s )

i m p l i e s t h e p r o b a b i l i t y i n e q u a l i t y ( 1 . 1 ) .

T h e p r o o f s o f b o t h l e m m a s a re e l e m e n t a r y a n d t h u s o m i t t e d .

R e m a r k 2 . 1 : I t i s w o r t h n o t i n g t h a t t h e s i m p le o b s e r v a t i o n o f t h e le m m a s c a n b e

a p p l i e d a l s o i n c a se s , w h e n - i n s t e a d o f t h e e x a c t v a l u e s i n ( 2 . 2 ) o r i n ( 2 . 4 ) - o n l y

r e s p e c t i v e a p p r o x i m a t i o n s 13 ', ~3 c a n b e p r o d u c e d f o r w h i c h g ( c , 13 ) ~ a o f f ( c , 13 ) ~


4/21

222 J. Pint6r

f (c , (3 ' ) 0) , 0 < a < 1 will be selected.

P r o p o s i t i o n 2 . 1 :

(Ap p l i ca t io n o f t h e o n e - s id ed C h eb y s h ev - i n eq u a li ty an d Lem m a 2 . 2 )

Assume tha t the s tanda rd dev ia t ion o f ~ , 8 = D~ a l so ex i s t s and i s know n (es t im ated) .

T hen P (~ ~> g + t)


5/21

Determin ist ic Ap proxim at ions of Probab i l i ty Inequa l it ies 223

P r o o f . ' A c c o r d i n g t o B e r n s t e in s e s t i m a t i o n t e c h n i q u e , w e c a n w r it e

P ( ~ - t l >~ t ) < ~ e ( h ) = e ( h , / ~ + t ) = e x p ( - h t ) E ( e x p h ( ~ - / ~ ) } O > 0 ) ( 2 . 1 0 )

T h e r e f o r e ( 1 . 1 ) e v i d e n t l y f o l l o w s b y ( 2 . 9 ) a n d ( 2 . 1 0 ) .

P r o p o s i t i o n 2 . 3 . ( A p p l i c a t io n o f t h e O k a m o t o - H o e f f d i n g i n e q u a l it y a n d L e m m a 2 .2 . )

A s s u m e t h a t w e k n o w t h a t t h e r e h o l d s l ~< ~ ~< u ( - - ~ < l < u < o o) , i .e . w e c a n ( o v e r )

e s t i m a t e t h e r a n g e o f t h e r . v . ~ . T h e n

(a - l ) In { (U -

l ) / ( a - l ) } + ( u - a )

In

{ ( u - U ) / ( u - a ) } < , ( u - l )

in c~

( l < t ~ < a < u )

( 2 . 1 1 )

i m p l i e s P ( ~ / > a ) ~< a .

P r o o f A c c o r d i n g t o H o e f f d i n g ( 1 9 6 3 ) , T h e o r e m 1 , f o r n i n d e p e n d e n t r . v . ' s ~ i,

0 ~< ~ i ~< 1 i = 1 , . . ., n , w i t h a r b i t r a r y 0 < t < 1 - # , t h e i n e q u a l i t y

P ( ~ ~ M + t ) < ( ~ / / J + t ) u + t [( 1 - M ) / ( 1 - / ~ - t ) ] l - u - t } n

( 2 . 1 2 )

ho s

T a k i n g n = 1 , f i rs t t r a n s f o r m t h e r .v . ~ i n t o t h e r a n g e [ 0 , 1 ] b y ~ := ( ~ -

l ) / ( u - l ) .

( B y t h i s , i n ( 2 . 1 2 ) / ~ a n d t a r e t o b e r e p l a c e d b y ( /~ - l ) / ( u - l ) a n d t / ( u - l ) , r e s p e c t i v e -

l y . N o w , w i t h a = t + / 1 ( 1 . 1 ) e a s i l y f o l l o w s b y ( 2 . 1 1 ) a n d ( 2 . 1 2 ) .

R e m a r k 2 . 3 :

A s i t is p r o v e d b y H o e f f d i n g ( 1 9 6 3 ) , i n e q u a l i t y ( 2 . 1 2 ) i s t h e b e s t t h a t c a n

b e o b t a i n e d ,

a p p l y i n g B e r n s te i n ' s m e t h o d ,

u n d e r t h e a s s u m p t i o n s o f h is T h e o r e m 1 .

( B e s t m e a n s t h a t t h e r e e x i s ts s u c h ( b i n o m i a l w i t h p a r a m e t e r ~ ) r .v . f o r w h i c h t h e

s ig n o f e q u a l i t y i s a t t a i n e d i n ( 2 .1 2 ) , c f. a l s o O k a m o t o 1 9 5 8 a n d P i n t 6 r 1 9 8 5 . )

P r o p o s i t i o n 2 . 4 :

( A p p l i c a t io n o f th e B e n n e t t - H o e f f d i n g i n e q u a l i ty a n d L e m m a 2 . 2. )

A s s u m e t h a t t h e r .v . ~ is b o u n d e d ( o n l y ) f r o m a b o v e : ~ ~< u a n d i t h a s v a ri a n c e 6 2 .


6/21

224 J. Pint4r

Then

[62 + (u - / l ) (a - / a ) ] In {1

+ ( u - I ~ ) (a - t i ) / 6 z } + (u - U ) ( u - a )

l n { 1 - ( a - l i ) / ( u - t ~ ) } ~ > [ ( u - / a ) e + 6 2 ] l n ( 1 / s ) ( / l < a K u )

2.13)

implies the rela tion P(~ >~ a) ~< ~.

P r o o f : Accord ing to Theorem 3 o f Hoe ffd ing (1963) , fo r indepe nden t r.v .' s ~ i , E~ i = O ,

~ i < ~ b , i = 1 , 2 , . . . , n

and fo r 0 < t < b the inequa l i ty

P(~ >t t) ~ {(1 +

b t / 6 2 ) - ( l + b t / ~ 2 ) ~ 2 / ( ~ 2 + ~ 2 ) ( 1 - t / b ) - ( 1 - t / b ) a 2 / ( b 2 + ~ 2 ) } n

(2 .14)

holds , w here (as earl ier ~ = ( ~

~ i ) /n . T a k i n g n = l , f i r s t t r a n s f o r m t h e r . v . ~ : = ~ -

to y ield E~ = 0 . ( th is impl ies tha t on the r ight-hand s ide the up per bo un d b is to be

rep laced by u - / l . ) Now , it i s eas ily seen tha t (2 .13) and (2 .14) lead to the inequal i ty

(1.1).

R e m a r k 2 . 4 :

As shown by Hoeffd ing (1963) , inequal i ty (2 .14 ) i s the best (cf . Rem ark

2 .3 ) tha t can be ob ta ined

a p p l y i n g B e r n s t e in ' s t e c h n i q u e -

an d k n o w i n g

o n l y

the

m e a n , v a r i a n c e

and an

u p p e r b o u n d

of ~.

(Note here tha t - as was po in ted ou t by one o f the Referees - (2 .6 ) i s

t h e s h a r p e s t

bou nd , i f / l ~< a < u ; fu r the r ,

- l)/ (a - l)


7/21

Deterministic Ap proximations of Probabil ity Inequalit ies 225

d e t e r m i n i s t i c r e l a t i o n s w h i c h i m p l y t h e p r o b a b i l i t y i n e q u a l i t y

P ~ > a ) = P ~ i~ > n a )~ < ~

( 3 . 1 )

F i r s t , t h e o n e - s i d e d C h e b y s h e v - i n e q u a l i t y a n d t h e t w o - s i d e d C h e b y s h e v - i n e q u a l i t y

a r e a p p l i e d f o r

a r b i t r a r y

( in c l ud i n g c o r r e l a te d a n d / o r u n b o u n d e d ) s u m m a n d s ; th i s w i ll

b e f o l l o w e d b y a p p li c a t io n s o f th e B e r n s t e in - ty p e e x p o n e n t i a l e s t i m a t e s f o r

i n d e p e n -

d e n t

a n d

b o u n d e d

( f r o m b o t h s i d e o r f r o m a b o v e ) s u m m a n d s .

A c c o r d i n g t o P r o p o s i t i o n 2 . 1 ( s e e (2 . 6 ) ), /1 +

6 X / ~ / a - 1 a ) ~ < a . I f t h e ~ i s a r e i n d e p e n d e n t , t h e n o b v i o u s l y 6 2 = 2 ; 6 ~ / n Z ;

t h e r e f o r e i n t h is c a s e ( 2 . 6 ) r e a d s a s

J l )

~i + n 6 ] 1 / a - 1 ) < . n a

( 3 . 2 )

i m p l y i n g ( 3 . 1 ) .

F u r t h e r , i n t h e c a s e o f s y m m e t r i c a l l y d i s t r i b u t e d s u m m a n d s ~ i , t h e B i e n a y m G

C h e b y s h e v i n e q u a l i t y y i e ld s

P ( I ( - / J [ ~> t ) = 2P(~ ~>/a + t ) ~~ O ,

i .e. ~ /l i + 6/2 /( 2 a ) ~ ~ + t ) < . E e x p { h [ ~ 1 ~ i - n ( l a + t ) ] I ( h > O )

( 3 . 4 )

N o w , i f t h e s u m m a n d s ~ i, i = 1 . . . . . n a r e i n d e p e n d e n t , t h e n

e x p f I t : , e x (

T h e r e f o r e i t r e m a i n s t o o b t a i n

s e p a r a t e

b o u n d s f o r th e e x p e c t e d v a l u e f a c t o r s in ( 3 . 5 )

a n d t h e n - w i t h t h e p u r p o s e o f s h a r p e n i n g t h e re s u lt - - t o m i n i m i z e th e a g g r e g a te d

r i g h t- h a n d s id e b o u n d w i t h r e s p e c t t o h . T h i s m e t h o d is a d v a n t a g e o u s , a s o p p o s e d t o

t h e s t r a ig h t f o r w a r d a p p r o a c h o f t r e a ti n g t h e s u m r .v . ~ i n t h e m a n n e r o f H o e f f d i n g

( 1 9 6 3 ) , w h e n t h e r . v . s ~ i a r e n o t i d e n t i c a l l y d i s t r i b u t e d a n d t h e i r r a n g e s , v a ri a n c e s ar e

d i f f e r e n t ( o r e v e n t h e a v a i l a b l e i n f o r m a t i o n m a y v a r y s e p a r a t e l y f o r t h e i n d i v i d u a l

r . v . s ~ i ) -

F o r e x a m p l e , l e t u s a s s u m e t h a t l i < ~ ~ i ~ U i , i = 1 . . . . n ( - - ~ < I < u i < ~ ) . B y t h e

c o n v e x i t y o f t h e f u n c t i o n e x p

( h x ) ,

fo r a r b i t ra ry l , x , u , l ~< x ~< u w e h ave

e x p

( h x ) < .

[ (u -

x ) / ( u -

l ) l e x p

( h l ) +

[ (x -

l ) / ( u -

l ) l e x p

( h u ) ( h > O )

Th er e f o r e w e have f o r t he r .v . s ~ i, i = 1 , . . . , n t he r e spec t i ve i nequa l i t ie s

E e x p

( h ~ i ) < . [ ( u i - i ) / ( u i - l i ) ]

e x p

( h l i )

+ [(/.ti - l i ) / ( u i - l i ) ] e x p ( h u i ) ( 3 . 6 )

S u b s t i t u ti n g ( 3 . 6 ) i n t o ( 3 . 5 ) w e o b t a i n

E e x p h ~ i - n 0 ~ + t ) = e x p

{ - h n ( l ~ +

t ) } I I e x p

( h ~ i )

I

~q

~ < e x p

{ - h n ( l a +

t )} 1 ] { [ ( u i - l a i ) / ( u

i - l i ]

e x p

( h l i )

?

+ [ ( U i - l i ) / ( U i - l i ) ] e x p ( h u i ) ) = e x p { - h n ( + t)} 1-I R i ( h )


9/21

Determin ist ic Ap proxim at ions of Proba bi l i ty Inequa l it ies 227

D e n o t i n g t h e l a s t e x p r e s s i o n o b t a i n e d b y

R ( h ) = R ( h , t , I J i , l i ,

b l i , i = 1 . . . . . n

w e h a v e

W e c a n s u m m a r i z e t h e a b o v e a r g u m e n t a t i o n in th e f o r m o f

P r o p o s i t i o n 3 . 1 : I f t h e r . v . s ~ i, i = 1 , . . . , n h a v e f i n i t e m e a n s / a i a n d r a n g e s [ l i , u i ] , t h e n

t h e i n e q u a l i t y

R ( h )

= e x p

{ - h n a } F I

1

{ [ ( u i -

U i ) / ( u i - l i ) ]

e x p

( h l i )

+ [(Ui - l i ) / ( u i - l i ) ] e x p ( h u i ) } < ~

( 3 . 7 )

w i t h a r b i t r a r y h > 0 im p l i e s t h e r e l a ti o n P ~ i > ~ n a < ~ ~ .

I n p a r t i c u l a r , t h e b e s t b o u n d o b t a i n a b l e b y B e r n s t e i n s m e t h o d i s g i v en b y R ( h * ) < ~ a ,

w h e r e h * s o l v e s R ( h * ) = r a i n R ( h ) .

h>O

W e sh a ll s h o r t l y d i sc u s s a ls o t h e c a s e , w h e n t h e a p p r o x i m a t i o n i s b / ls e d o n t h e

k n o w l e d g e o f m e a n , v a r ia n c e a n d u p p e r b o u n d v a l u e s. I n B e n n e t t ( 1 9 6 2 ) t h e i n -

e q u a l i t y

E e x p ( h ~ ) ~ < [ b 2 / ( b 2 + 6 2 ) ] e x p ( - g 2 h / b ) + [ ~ 2 / ( b 2 6 2 ) ] e x p ( h b ) ( 3 . 8 )

is p r o v e d f o r a r b i t r a r y h > 0 , w h e r e b y s u p p o s i t i o n E ~ = 0 , E ~ 2 = 6 2 a n d ~ ~< b . A p p l y -

i n g t h i s r e l a t i o n f o r t h e r . v . s

~ i - f J i , i = 1 , . . . , n ( ~ i - l a i < ~ u i - I a i )

w e o b t a i n

E e x p

{ h ( ~ i - l a i ) ) < ~ ( ( u i - i J i ) 2 / [ ( u i - U i )

2 + 6 2 ] } e x p

{ - h 6 ] / ( u i - i ) }

+ ~ 6 ] / [ ( t t i _ / / i )2 + 6 ] ] } e x p { h ( u i - / . t i ) } = S i ( h )


10/21

228 J. Pint6r

T h e r e f o r e i n t h i s c a s e w e o b t a i n t h e r e l a t i o n s

P ~i >~ ~ #i + n t


11/21

Determin ist ic Ap prox im at ions o f Probab i l i ty Inequa li ties 229

Remark 3.3. F o l l o w i n g H o e f f d i n g ( 1 9 6 3 ) , t h e c as e o f dependent s u m m a n d s is e a s il y

d e d u c i b l e t o t h e p r e v i o u s a n a l y s i s . A s s u m e t h a t t h e r . v . T i s a s u m o f t h e f o r m

N N

T= ~ p iT] p]>~O ~, p l= l

( 3 . 1 0 )

1 1

w h e r e e a c h

T]

i s a s u m o f i n d e p e n d e n t r .v . 's n o t e t h a t t h e T i 's n e e d

not

t o b e i n d e p e n -

d e n t . ( E v i d e n t l y , t h e c a s e o f d e p e n d e n t s u m m a n d r . v .' s c a n b e i n t e r p r e t e d a s a s p e c ia l

c a se o f ( 3 . 1 0 ) . ) B y B e r n s t e i n 's i n e q u a l i t y w e h a v e

P (T > / t )< ~ e x p ( -h t )E e x p (h T ) (h > 0 )

F u r t h e r , b y th e c o n v e x i t y o f t h e e x p o n e n t i a l f u n c t i o n o n e c a n a p p l y J e n s e n ' s i n e q u a li -

t y f o r a r b i t r a r y a r g u m e n t s t = N pit] t h i s g iv e s

N )N

x p ( h t ) = e x p h ~ pl ti ~ 1 p j e x p ( h tj )

T h e r e f o r e t a k i n g th e e x p e c t a t i o n o f e x p { h ( T - t ) } w e o b t a i n

N N

P ( T > / t )< . X p i e e x p { h ( T j - t ) } = s p /Q /

H e r e t h e s u m m a n d f a c to r s Q j - d e f i n e d b y s u m s o f independent r . v . ' s c a n b e

b o u n d e d s e p a r a t e l y , a p p l y i n g P r o p o s i t i o n s 3 .1 a n d 3 .2 .

N o t e f i n a ll y t h a t t h e f r a m e w o r k o f S e c t i o n 3 o b v i o u s l y a p p l i e s t o a ll l i n ea r c o m -

n

b i n a t i o n s ~ = 23

e i ~ i

w h e r e t h e ei 's a r e g i v e n r e a l c o e f f i c i e n t s .

S o m e P o s si b le A p p l ic a t i o n s

T h e e x a m p l e s s h o w n b e l o w s e rv e p r i m a r i l y f o r th e p u r p o s e o f i l lu s t r at i n g t h e p o t e n -

t ia ls o f t h e s u g g e s t e d a p p r o a c h : h e n c e , d e t ai ls u n n e c e s s a r y i n t h e p r e s e n t c o n t e x t w i ll

b e o m i t t e d o r o n l y s h o r t l y r e f e r re d t o .


12/21

230 J . Pint6r

4 .1 G e n e r a l i ze d ( S t o c h a s ti c ) K n a p s a c k P r o b l e m

T h e s o- c al le d k n a p s a c k p r o b l e m ( K P ) is a ( li n ea r ) i n te g e r p r o g r a m m i n g p r o b l e m

w i t h a s i n g l e i n e q u a l i t y c o n s t r a i n t :

m a x c T x c C R~ )

d T x < . b d E R ~ , b > O )

( 4 . 1 )

x T = x I , . . . , X n ) x i = 0 o r l , i = 1 . . . . . n

T h e c o n c e p t w h i c h l e a ds t o th e m o d e l ( 4 . 1 ) c a n b e v e r b a l ly s t a t e d e .g . i n t h e f o l l o w i n g

w a y : f i n d a p o r t f o l i o o f p o s si b l e y e s / n o a c t i o n s s u c h t h a t i t s s u m m e d b e n e f i t s ( p o s i ti v e

i m p a c t s ) a r e m a x i m a l , w h i le i t s t o t a l c o s t s ( r e so u r c e d e m a n d s ) d o n o t e x c e e d a p r e f i x e d

u p p e r b o u n d . A m o n g t h e m a n y a p p l i c a ti o n s o f t h is m o d e l , w e m e n t i o n o n l y t he r e c e n t

w o r k o f L o o t s m a , M e i s n e r a n d S c h e l l e m a n s ( 1 9 8 6 ) , in w h i c h t h e m u l t i o b j e c t i v e p r o b -

l e m o f n a t i o n a l e n e r g y r e s e a r c h a n d d e v e l o p m e n t p l a n n i n g is r e d u c e d - v i a w e i g h i n g

s c a l a r i z a t i o n o f t h e o b j e c t i v e s - t o th e f o r m ( 4 . 1 ) .

N o w , i f - i n s t e a d o f a c c e p t i n g f i x e d v a l u es e i > 0 - w e a s s u m e t h a t t h e b e n e f i t s

a r e u n c e r t a i n t o s o m e e x t e n t , t h e n t h e y c a n b e m o d e l l e d b y i n t r o d u c i n g r .v . 's rT i,

i = 1 , . . . , n. I n t h i s c a s e i t m i g h t b e r e a s o n a b l e t o s e l e c t p o r t f o l i o s w h i c h assure a t l eas t

s o m e f i x e d t o t a l r e t u r n i n a p r o b a b i l i s t ie s e n se . T h i s i d e a m a y l e a d t o t h e f o l l o w i n g

s t o c h a s t i c e x t e n s i o n o f t h e K P (4 . 1 ) :

m a x P r f x >~ a ) ( a > 0 e x t e r n a l l y g i v en m o d e l p a r a m e t e r ) ( 4 . 2 )

dTx ~b

w i t h t h e i n t e g r a l i t y a s s u m p t i o n s x i = 0 o r 1 . I n t h e f o l l o w i n g d e t e r m i n i s t i c a p p r o x i m a -

t i o n s t o t h e o b j e c t i v e f u n c t i o n o f ( 4 . 2 ) w il l b e d e ri v e d , w h e n a d d i t i o n a l i n f o r m a t i o n

c o n c e r n i n g t h e r . v . ' s r/i i s a v a i l a b l e . ( N o t e t h a t - a s t h e r e t u r n s o f t e n h a v e s t r o n g l y

s k e w e d p . d .f . ' s - t h e a p p r o x i m a t i o n i n d u c e d b y th e c e n tr a l l im i t t h e o r e m m i g h t be

r a t h e r c r u d e f o r s m a l l v a l e s o f n , w h i le t h e a n a l y t i c a l e x p r e s s i o n o f t h e p . d . f, o f r / r x =

~ , p i x i - e v e n f o r f i x e d v a l u e s x i - is u n k n o w n i n m a n y c a se s .)

1

B y t h e r e s u l ts p r e s e n t e d e a r li e r, w e k n o w s e v e ra l i n e q u a l i ti e s o f th e f o r m

P ~ > i a )


13/21

Deterministic Approxim ations of Probability Inequalities 231

{ P B }

b e i n g a j o i n t s y m b o l i c n o t a t i o n fo r t h e p ro b ab i l i t y b o u n d s o b t a i n ed . Hen ce , i t

is r ea s o n ab le t o fo l l o w t h e t w o -s t ep p ro ced u re b e l o w:

i) Select f i rs t the a d a p t i v e p a ram e t e r s o f { P B ) in such a way th a t { P B } is m i n i m i z e d .

This ac t ion makes the bo und sha rpes t , imp ly ing tha t P(~ ~>a) and { P B } are

as close as pos s ible in the g iven fram ew ork . (Du ring th is s tep the x i s are regarded

as ex o g en o u s l y f i x ed p a ram e t e r s , t h e re fo re t h e b o u n d i n g p ro ced u re s h o u l d b e

val id for their

al l

feas ible set t ings . )

i i) R e p la c e t h e p r o b l e m o f m a x i m i z in g P ( ~ / > a ) b y

m a x i m i z i n g

{PB}: th is act ion is

b as ed o n t h e

q u a n t i t a t i v e i n f o r m a t i o n

ab o u t t h e co m p o n e n t s ~ i o f ~ ( t hi s i n fo rm a-

t i o n d e t e rm i n es an ap p r o x i m a t e l y b e s t s e t ti n g o f t h e

d e c i s io n v a r ia b le s

{xi}) .

I f thi s c onc ep t i s app l ied to the r .v . ~ = r S x = 2; 7 7 ix i , then the ob jec t ive func t io n in

I

(4 . 2 ) w i ll b e r ep l aced ( s y m b o l i ca ll y ) b y d e t e rm i n i s ti c ex p re s si o n s o f t h e fo rm

m a x

{ P B ( x ) }

(4 .4 )

T h e c o n c r e t e f o r m o f

{ P B ( x ) }

d ep en d s , o f co u r s e , o n t h e q u an t i t a t i v e i n fo rm a t i o n

ab ou t the r /i 's . Be low we shal l g ive exam ples o f the ob jec t ive func t ion (4 .4 ) , app ly ing

some resu l ts o f Sec t ion 3 .

Assume f i r s t tha t fo r a l l r .v . 's rl i we kn ow (o r can es t im ate) the i r low er /u ppe r

b o u n d s an d m ean . B y P ro p o s i t io n 3 . 1 , fo r an a rb i t r a ry s u b s e t o f in d ice s I C { 1 . . . . . n }

and in dep end en t r .v . 's ~ i i E I wi th kn ow n range and me an , we have

P ~ ~ i >~ a ) ~< ex p { - h a } F I { [ u i - t ~ i ) / u i - l i ) e x p h l i )

i E l i ~ I

+ [(/.zi -

l i ) / (u i - l i ) ]

e x p

( h u i ) } .

Th erefo re fo r a ll index-se t s I ch arac te r ized by

x i = 1 i E I ,

we have

l n P ~ i x i > ~ a = l n P ( G ~ i > ~ a ) (4 .5 )

iE

< ~ - h a

+ 2 ; I n { [ u / - U i ) / u i - l i ) ] ex p

( h l i )

i ~ l

+ [(/ai -

l i ) / (u i - l i )]

e x p

( h u i ) }


14/21

232 J. Pint6r

l l

= - h a + ~ i n { [ ( u i - U i ) / ( u i - l i ) ] e x p ( h l i )

1

+ [ ( # i - [ i ) / ( u i - l i ) ] e x p ( h u i ) } x i

K e e p i n g n o w t h e p a r a m e t e r v a l ue s a a n d h > 0 f i x e d ( i .e . a l r e a d y s e le c t e d ) , f r o m ( 4 . 5 )

w e c a n c o n c lu d e t h a t m a x i m i z i n g t h e p r o b a b i l i t y b o u n d

{ P B ( x ) }

i n ( 4 . 4 ) l e a d s t o t h e

m a x i m i z a t i o n o f t h e s u m i n t h e la s t e x p r e s s i o n o f ( 4 .5 ) . I n o t h e r w o r d s , t h e s t o c h a s t i c

K P ( 4 . 2 ) is a p p r o x i m a t e d b y t h e f o l l o w i n g d e t e r m i n i s t i c K P :

n

m a x Y , c } 1 )x i

d T x ~ b

( 4 . 6 )

w h e r e

c} 1) = in { [ ( u i -

t J i ) / ( u i - l i) ]

e x p

( h l i ) +

[ (g i -

l i ) / ( u i

- l i ) ]

e x p

h u i ) }

a n d

x i = O o r 1, i = 1 , . . . , n .

N o t e t h a t t h is d e t e r m i n i s t ic r e f o r m u l a t i o n o f p r o b l e m ( 4 . 2 ) i s p e r f e c t , w h e n i n ( 3 . 6 )

e q u a l i t y s t a n d s ( i. e. a l l r . v . 's r/i h a v e t w o - p o i n t d i s c r e t e d i s t r i b u t i o n s w i t h

P ( ~ i = l i ) =

( u i - # i ) / ( u i - l i ) a n d P ( ~ i = u i ) = ( g i - l i ) / ( u i - l i ) ) . N o t e a l so t h a t a l t h o u g h t h e e x o g e n -

o u s l y g i ve n a s p i r a t io n l e v e l a d o e s n o t o c c u r e x p l i c i t l y i n t h e o b j e c t iv e f u n c t i o n o f

p r o b l e m ( 4 .6 ) , t h e p a r a m e t e r v al u e h ( w h i c h m i n i m i z e s t h e b o u n d ) d e p e n d s o n a ;

h e n c e , t h e c o e f f i c i e n t s c ~ 1) a r e im p l i c i t f u n c t i o n s o f a . W e e m p h a s i z e t h a t h h a s t o

b e f i x e d i n d e p e n d e n t l y o f t h e s e l e c t i o n o f th e i n d e x - s e t I ( x i = 1 , i E / ) , f o r t h e s a k e o f

c o m p u t a t i o n a l s i m p l i c i t y : t h e r e f o r e I = I o = ( 1 . . . . . n } se e m s t o b e t h e m o s t n a t u r a l

c h o i s e . L e t u s n o t e f i n a l l y t h a t i n t h e c a s e o f u i = l i = g i = c i , i = 1 , . . . , n ( i . e . w h e n

o u r r .v . 's r e d u c e t o e x a c t p o i n t e s t i m a t e s

c i

o f t h e b e n e f i t s ) w e h a v e c~ l ) =

h c i ( h >

0 ) :

t h is w a y , t h e a p p r o x i m a t i n g s t o c h a s t i c K P f o r m u l a t i o n ( 4 . 6 ) i n c l u d e s a s a s p e c ia l c a se

t h e i n i t i a l , d e t e r m i n i s t i c K P .

A n a l o g o u s c o m m e n t s a r e v a li d a l so i n t h e c a s e , w h e n t h e m e a n /~ g , u p p e r - b o u n d u i

a n d v a r i an c e 6 i p a r a m e t e r s a r e k n o w n / e s t i m a t e d f o r a ll ~i 's . W i t h o u t r e p e a t i n g t h e

p r e v io u s a r g u m e n t a t i o n , o n e c a n s ee t h a t t h e s t o ch a s t ic K P ( 4 .2 ) ca n b e a p p r o x i m a t e d

b y


15/21

Determin ist ic Ap prox imat io ns o f Proba b i l i ty Inequa li ties 233

i

m a x O } 2 ) X i

d T x ~ b

( 4 . 7 )

w h e r e

e} 2) : In I u i - I R i ) 2 / [ u i - / 2 i ) z + 6 2 1 e x p [ - h 6 ~ / u i - /~ i )1

+ 6 2 / [ u i - I Ji ) 2 + 6 2

] ) e x p

[ h u i - / l i ) ]

i +

ht2 i

a n d

x i = O

o r l , i = l , . . . , n .

A s e a rl i e r , t h e a s p i r a t i o n l e v e l a i s i m p l i c i t l y i n c l u d e d v i a t h e p a r a m e t e r h . B e si d e s, i t

i s e a s y t o s e e a g a i n t h a t

c ~ 2 ) - ~ h l J i , i = 1 , . . . , n ,

w h e n

u i - ~ l~ i

a n d 6 i ~ 0 .

C o n c l u d i n g t h i s e x a m p l e w e n o t e t h a t - a s i t is w e l l k n o w n - ( s u b ) o p t i m a l s o lu -

t i o n s t o t h e d e t e r m i n i st i c K P ca n b e e f f ic i e n c tl y g e n e r a t e d , f o r m i n g t h e m o n o t o n o u s l y

d e c r e a s in g o r d e r o f b e n e f i t / c o s t r a t io s c i / d i } . T h u s , a s i m i la r p r o c e d u r e c a n b e a p p l i e d

t o t h e d e t e r m i n i s t i c a p p r o x i m a t i n g p r o b l e m s (4 . 6 ) , ( 4 . 7 ) : t h i s m a y b e h e l p f u l e .g . i n

e x p l o r i n g t h e v a r i a t i o n s o f t h e p r o j e c t r a n k s a s a f u n c t i o n o f th e t a r g e t le v e l a .

4.2 Chance C onstrained Linear Programming

T h e li n ea r p r o g r a m m i n g p r o b l e m ( e.g . f o r f in d i n g th e m i n i m u m - c o s t p r o d u c t i o n p l a n

w h i c h s a t is f ie s d i f f e r e n t t y p e s o f d e m a n d s ) c a n b e s t a t e d a s

m i n cTx

a T x > / b i i = 1 , . . . , m

x E R ~ _ , a i , c E R n )

4 .8 )

T h e f o r m u l a t i o n ( 4 . 8 ) m a y h a v e d i f f e r e n t e x te n s i o n s , w h e n p o s s ib l e u n c e r t a i n t i e s ,

c o n c e r n i n g t h e c o s t - c o e f f i c i e n t s

c ] ] = 1 , . . . , n ,

p r o d u c t i o n t e c h n o l o g y c o e f f i c i e n t s


16/21

234 J. Pint6r

a q a i = ( a n . . . . , a i n ) , i = 1 . . . . . m ) and demand leve l s b i i = 1 . . . . . m , are a l so t aken

in to cons idera t ion . A poss ib le s tochas t ic re fo rmula t ion o f (4 .8 ) has the fo l lowing

f o r m :

m i n E

E c j ) x j

P ~ a g x j > b i ) > t 1 - e i i = l , . . . , m

0 < e i < 1 p a ram e t e r s )

(4 .9 )

Acco rd i n g t o (4 . 9 ) , t h e ex p ec t ed (p ro d u c t i o n ) co s t s a r e t o b e m i n i m i zed , wh i l e t h e

sa t i s fac tion o f the (dem and ) cons t ra in t s is to be assu red wi th respec t ive , su f f ic ien t ly

h igh p robab i l i t i es . No te tha t (4 .9 ) i s n o t t h e u n i q u e o r b e s t s t o ch as t i c v e r si o n

of (4 .8 ) : i t s use can be reasonab le e .g . in such s i tua t ions , when the vec to rs a i can be

cons idere d as (ap pro x im ate ly ) indep ende n t , wh i le there a re s ign i fican t d i f fe rences be-

t ween t h e i m p o r t an ces o f m ee t i n g t h e d em an d s b i . Th erefo re (4 .9 ) can be a pp l ied e .g .

in mo del l ing cer ta in a l loca t ion /d i s t r ibu t ion p rob lem s , when exp l ic i t p r io r i ty o rde r

be tween the d i f fe ren t demands i s to be cons idered .

Assuming tha t the mean va lues E c ] ) a re k n o wn (o r p ro p e r l y e s t i m a t ed ) , t h e o b -

jec t ive func t ion o f (4 .9 ) i s aga in l inear . Below we p resen t exp l ic i t approx imat ions o f

the separa te chance -cons t ra in t s , on the bas is o f the p rev ious es t imates . (Note th a t

ear l ie r S inha 1963 app l ied the Bienaym &C hebyshev inequ al i ty wi th the same ob jec t ive ,

see also Wets 1983.)

Def ine

~ i o = b i i = l . . . . . m , ~ i] = - a i i j = l , . . . , n , i = l , . . . , m ,

fu r t h e r o n , l e t

x o = l . T h e n w e h a ve t h e t r a n s c r i b e d p r o b a b i l i s t i c c o n s t r a i n t s P ( o ~ ~ i l x j < < , O ) > l - e i ,

i.e.

P ~ o ~ i l x i > ~ O ) < ~ e i i = 1 . . . . . rn ( 4 . 1 0 )

(by assu mp t ion , a l l occur ing r.v .' s have co n t inu ous p .d . f .' s ) .

By Propo s i t ion 2 .1 we know tha t (app ly ing the one-s ided Chebyshev- ineq ual i ty )

the re la t ion

/1 + 8 X /1/a - 1 ~ a im plie s P(~ >~ a) ~

Documents

Pinter89 Probabilistic Inequalites