pinter89 probabilistic inequalites
TRANSCRIPT
-
8/10/2019 Pinter89 Probabilistic Inequalites
1/21
Z O R M e t h o d s a n d M o d e ls o f O p e r a ti o n s R e s e a rc h ( 1 9 8 9 ) 3 3 : 2 1 9 - 2 3 9
D eterm inist ic pprox ima tions of Prob abi l ity Inequa l it ies
B y J . P i n t 6 r 1
A b s t r a c t : A s i m p le g e n e r a l f ra m e w o r k f o r d er i v in g exp l i c i t d e t e r m i n i s t i c a p p r o x i m a t i o n s o f p r o b a -
b i l i t y i n e q u a l i ti e s o f t h e f o r m P ( ~ / > a ) ~< ~ i s p r e s e n t e d . T h e s e a p p r o x i m a t i o n s a r e b a s e d o n l i m i t e d
p a r a m e t r i c i n f o r m a t i o n a b o u t t h e i n v o l v e d r a n d o m v a r ia b l es ( su c h a s th e i r m e a n , v a r i a n c e , r a n g e o r
u p p e r b o u n d v a l u e s ). F i r s t t h e c a s e o f a s i n g le r a n d o m v a r i a b l e ~ i s a n a l y s e d , f o l l o w e d b y t h e c a s e s
n
o f i n d e p e n d e n t a n d d e p e n d e n t s u m m a n d s f = ~ ~ i. A s e x a m p l e s o f p o ss i b le a p p l ic a t i o n s , a st o c h a s -
1
t i c e x t e n s i o n o f t h e " k n a p s a c k p r o b l e m " a n d t h e s t o c h a s t i c l i n e a r p r o g r a m m i n g p r o b l e m w i t h
s e p a r a te c h a n c e - c o n s t r a i n t s a r e i n v e s ti g a t e d : w e p r o v i d e a p p r o x i m a t e d e t e r m i n i s t i e su r r o g a te s f o r
t h e s e p r o b l e m s .
Z u s a m m e n f a s s u n g : E s w i r d e i n R a h m e n z u r A b l e i t u n g e x p l i z it e r d e t e r m i n i s t i s c h e r A p p r o x i m a t i o n
f ii r W a h r s c h e i n l i c h k e i t su n g l e i c h u n g e n d e r F o r m P ( ~ ~ a ) ~< a a n g e g e b e n . D i es e A p p r o x i m a t i o n e n
b a s i e r en a u f b e g r e n z t e r p a r a m e t r i s c h e r I n f o r m a t i o n t ib e r d ie b e t e i l ig t e n Z u f a l ls v a r ia b l e n ( w ie i h r
E r w a r t u n g s w e r t , V a r i a n z , W e r t e b e r e ic h o d e r o b e r e S c h r a n k e n ) . Z u e r s t w i r d d e r F a i l e i n e r Z u f a l ls -
n
v a ri a b le n f a n a l ys i er t, s o d a n n w e r d e n S u m m e n v o n u n a b h g n g i g e n S u m m a n d e n ~ = Y, ~ i b e t r a c h t e t .
i=1
A l s B e i s pi e le f ti r m 6 g l ic h e A n w e n d u n g e n w i r d e i n e s to c h a s t i s c h e E r w e i t e r u n g d e s R u c k s a c k -
p r o b l e m s u n t e r s u c h t s o w i e s t o c h a s t i s c h e l i n e a r e P r o g r a m m e m i t s e p a r a b l e n W a h r s c h e i n l i c h k e i t s -
r e s t r i k t i o n e n . F i i r d i e s e P r o b l e m e w e r d e n n / i h e r u n g s w e i s e d e t e r m i n i s t i s c h e E r s a t z p r o b l e m e a n g e -
g e b e n .
K e y w o r d s c h a n c e - c o n s t r a i n t s ; p r o b a b i l i t y i n e q u a l i t i e s; e x p l i c it d e t e r m i n i s t i c a p p r o x i m a t i o n s .
1 J ~t no s P i n t 6 r , R e s e a r c h C e n t e r f o r W a t e r R e s o u r c e s D e v e l o p m e n t ( V I T U K I ) , 1 4 5 3 B u d a p e s t ,
P . O . B o x 2 7 , H u n g a r y .
0 3 4 0 9 4 2 2 / 8 9 / 4 / 2 1 9 2 3 9 $ 2 . 5 0 9 1 9 8 9 P h y s ic a -V e r la g , H e i d e l b e r g
-
8/10/2019 Pinter89 Probabilistic Inequalites
2/21
220 J. Pint6r
1 I n t r o d u c t i o n
L e t ~ = ~ (w) b e a r an d o m v a ri ab le ( r .v . ) w i th c o n t i n u o u s , s t r i c t ly m o n o to n o u s p ro b a -
b i l i ty d i s t r ibu t ion func t ion (p .d . f . ) . We sha l l p resen t a s imple genera l approach to ap -
p ro x im a te p ro b ah i l i t y i n eq u al it ie s o f t h e fo rm
P ~ a ) ~ a (1 .1 )
where --oo< a < ~ , 0 ~< a ~< 1 are g iven values. The ap pro xim atio ns are at ta ine d via
subs t i tu t ing (1 .1 ) by (genera l ly speak ing , s t r i c te r ) de te rm in is t i c inequal i ti es which con-
t a in p a ram e te r s t h a t
partially
charac te r ize the p .d . f , o f ~ (such as i t s mean , var iance ,
range e tc . ) . To mot iva te th i s s tudy , l e t us men t ion tha t e .g . in s tochas t ic (chance-
cons t ra ined) p rogramming , r i sk ana lys i s , re l i ab i l i ty theory e tc . one o f ten fo rmula tes
cer ta in s ta t i s t i ca l requ i rements as p robab i l i s t i c cons t ra in t s which can be b rough t to
fo rm (1 .1 ) , see e .g . Berger (1985 ) , D emp s ter (1980) , Wets (1983) . F req uen t ly , the r .v .
depends on severa l dec i s ion var iab les p lus a number o f (uncon t ro l lab le , unknown)
ran d o m fac to r s i n a co m p l i ca t ed , an a ly t ica l l y n o n - tr ac t ab l e ( im p l i c i t) m an n e r : co n s e -
q u en t l y , ev en n o t ev e ry accu ra t e , b u t
explicit
d e t e rm in i s t i c ap p ro x im a t io n s o f (1 . 1 )
m a y b e o f p rac t i ca l v a lu e .
There ex is t s a fa i r ly ex tens ive l i t e ra tu re conc ern ing d i f fe ren t p robab i l i s t i c inequali -
t ies which - w i th respec t ive spec i f ica t ions o f the i r r .v . 's - y ie ld sharp boun ds , see fo r
ex am p le B ah ad u r an d R ao (1 9 6 0 ) , B en n e t t (1 9 6 2 ) , C h e rn o f f (1 9 5 2 ) , Daws o n an d
San k o f f (1 9 6 7 ) , Du p aco v a (1 9 8 0 , 1 9 8 7 ) , Ga l am b o s (1 9 7 7 ) , G o d win (1 9 5 5 ) , Ho e f fd in g
(1963 ) , Huang , Z iem ba and Ben-Tal (1977) , K al l and S toya n (1982 ) , Kankov a (1977) ,
Kle in Hanev .e ld (1985) , Kwere l (1975) , Madansky (1960) , M6ri and Sz6kely (1985) ,
Ok am o to (1 9 5 8 ) , Pe rcu s an d Pe rcu s (1 9 8 5 ) , P l a t z (1 9 8 5 ) , P in t6 r (1 9 8 5 ) , P ro h o ro v
(1959) , Sa the , Pradhan and Shah (1980) , Seppala (1975) , S inha (1963) , Sz~n ta i
(19 85) and Yudin (1980) . In th i s pape r - based on severa l c i t ed re ference s - m ean /
range , mean /var iance and me an /var iance / ran ge app rox im at ions w i ll be g iven . The un i -
var ia te case is fo l lo we d b y the mo re genera l case o f sums o f r .v . 's . Tw o possib le appl i-
cat ions serve as i llus trat ive exam ples: a s toch ast ic vers ion o f the so-cal led kna psa ck
p ro b l em o f i n t eg e r p ro g ram m in g an d t h e s t o ch ast i c l in ea r p ro g ram m in g p ro b l em w i th
separa te chance -cons t ra in t s .
2 T h e C a s e o f a S i ng le R a n d o m V a r i a b l e
Cons ider the p rob ab i l i ty inequ al i ty (1 .1 ) w i th g iven para m eter va lues a and a : the ob-
serva tions be low p erm i t the der iva t ion o f
different deterministic surrogates
fo r (1 .1 ) .
-
8/10/2019 Pinter89 Probabilistic Inequalites
3/21
Determ inistic Ap proxim ations o f Probab ility Inequalities 221
L e m m a 2 .1 : S u p p o s e t h a t a p r o b a b i l i s t i c r e l a t i o n
P ( ~ > I f ( c , ~ ) ) < , g ( c , ~ )
( 2 . 1 )
c a n b e g i v e n f o r t h e r . v . ~ ( a n d a l l v a l u e s o f 1 3), w h e r e 13 s y m b o l i z e s s o m e p a r a m e t e r o f
t h e i n e q u a l i t y ( 2 . 1 ) , c = c~ r e p r e s e n t s q u a n t i t a t i v e i n f o r m a t i o n a b o u t ~ ( s u c h a s i ts
m e a n , v a r i an c e , h ig h e r m o m e n t s , r a n g e e t c .) , f a n d g ar e k n o w n c o n t i n u o u s f u n c t i o n s
o f th e i r a rg u m e n t s . I f - f o r s o m e ad m i s si b le p a r a m e t e r i z a t io n s - t h e e q u a t i o n g( c, [3) = a
h a s a s o l u t i o n i n / 3 :
= k ( c , a ) ( 2 . 2 )
t h e n t h e e x p l i c i t d e t e r m i n i s t i c r e l a t i o n
f c , k c , ~ ) ) < ~ a 2 .3 )
i m p l i e s t h e p r o b a b i l i t y i n e q u a l i t y ( 1 . 1 ) .
L e m m a 2 .2 : A s s u m e a g ai n t h a t t h e p r o b a b i l i t y i n e q u a l i t y ( 2 . 1 ) h o ld s a n d s u p p o s e t h a t
f o r s o m e p a r a m e t r i z a t i o n s t h e e q u a t i o n f ( c , 13) = a h a s a s o l u t i o n i n 1 3 :
13 = h( c, a) ( 2 . 4 )
T h e n t h e e x p l i c i t d e t e r m i n i s t i c r e l a t i o n
g c , h c , a ) ) < ,~ 2 . s )
i m p l i e s t h e p r o b a b i l i t y i n e q u a l i t y ( 1 . 1 ) .
T h e p r o o f s o f b o t h l e m m a s a re e l e m e n t a r y a n d t h u s o m i t t e d .
R e m a r k 2 . 1 : I t i s w o r t h n o t i n g t h a t t h e s i m p le o b s e r v a t i o n o f t h e le m m a s c a n b e
a p p l i e d a l s o i n c a se s , w h e n - i n s t e a d o f t h e e x a c t v a l u e s i n ( 2 . 2 ) o r i n ( 2 . 4 ) - o n l y
r e s p e c t i v e a p p r o x i m a t i o n s 13 ', ~3 c a n b e p r o d u c e d f o r w h i c h g ( c , 13 ) ~ a o f f ( c , 13 ) ~
-
8/10/2019 Pinter89 Probabilistic Inequalites
4/21
222 J. Pint6r
f (c , (3 ' ) 0) , 0 < a < 1 will be selected.
P r o p o s i t i o n 2 . 1 :
(Ap p l i ca t io n o f t h e o n e - s id ed C h eb y s h ev - i n eq u a li ty an d Lem m a 2 . 2 )
Assume tha t the s tanda rd dev ia t ion o f ~ , 8 = D~ a l so ex i s t s and i s know n (es t im ated) .
T hen P (~ ~> g + t)
-
8/10/2019 Pinter89 Probabilistic Inequalites
5/21
Determin ist ic Ap proxim at ions of Probab i l i ty Inequa l it ies 223
P r o o f . ' A c c o r d i n g t o B e r n s t e in s e s t i m a t i o n t e c h n i q u e , w e c a n w r it e
P ( ~ - t l >~ t ) < ~ e ( h ) = e ( h , / ~ + t ) = e x p ( - h t ) E ( e x p h ( ~ - / ~ ) } O > 0 ) ( 2 . 1 0 )
T h e r e f o r e ( 1 . 1 ) e v i d e n t l y f o l l o w s b y ( 2 . 9 ) a n d ( 2 . 1 0 ) .
P r o p o s i t i o n 2 . 3 . ( A p p l i c a t io n o f t h e O k a m o t o - H o e f f d i n g i n e q u a l it y a n d L e m m a 2 .2 . )
A s s u m e t h a t w e k n o w t h a t t h e r e h o l d s l ~< ~ ~< u ( - - ~ < l < u < o o) , i .e . w e c a n ( o v e r )
e s t i m a t e t h e r a n g e o f t h e r . v . ~ . T h e n
(a - l ) In { (U -
l ) / ( a - l ) } + ( u - a )
In
{ ( u - U ) / ( u - a ) } < , ( u - l )
in c~
( l < t ~ < a < u )
( 2 . 1 1 )
i m p l i e s P ( ~ / > a ) ~< a .
P r o o f A c c o r d i n g t o H o e f f d i n g ( 1 9 6 3 ) , T h e o r e m 1 , f o r n i n d e p e n d e n t r . v . ' s ~ i,
0 ~< ~ i ~< 1 i = 1 , . . ., n , w i t h a r b i t r a r y 0 < t < 1 - # , t h e i n e q u a l i t y
P ( ~ ~ M + t ) < ( ~ / / J + t ) u + t [( 1 - M ) / ( 1 - / ~ - t ) ] l - u - t } n
( 2 . 1 2 )
ho s
T a k i n g n = 1 , f i rs t t r a n s f o r m t h e r .v . ~ i n t o t h e r a n g e [ 0 , 1 ] b y ~ := ( ~ -
l ) / ( u - l ) .
( B y t h i s , i n ( 2 . 1 2 ) / ~ a n d t a r e t o b e r e p l a c e d b y ( /~ - l ) / ( u - l ) a n d t / ( u - l ) , r e s p e c t i v e -
l y . N o w , w i t h a = t + / 1 ( 1 . 1 ) e a s i l y f o l l o w s b y ( 2 . 1 1 ) a n d ( 2 . 1 2 ) .
R e m a r k 2 . 3 :
A s i t is p r o v e d b y H o e f f d i n g ( 1 9 6 3 ) , i n e q u a l i t y ( 2 . 1 2 ) i s t h e b e s t t h a t c a n
b e o b t a i n e d ,
a p p l y i n g B e r n s te i n ' s m e t h o d ,
u n d e r t h e a s s u m p t i o n s o f h is T h e o r e m 1 .
( B e s t m e a n s t h a t t h e r e e x i s ts s u c h ( b i n o m i a l w i t h p a r a m e t e r ~ ) r .v . f o r w h i c h t h e
s ig n o f e q u a l i t y i s a t t a i n e d i n ( 2 .1 2 ) , c f. a l s o O k a m o t o 1 9 5 8 a n d P i n t 6 r 1 9 8 5 . )
P r o p o s i t i o n 2 . 4 :
( A p p l i c a t io n o f th e B e n n e t t - H o e f f d i n g i n e q u a l i ty a n d L e m m a 2 . 2. )
A s s u m e t h a t t h e r .v . ~ is b o u n d e d ( o n l y ) f r o m a b o v e : ~ ~< u a n d i t h a s v a ri a n c e 6 2 .
-
8/10/2019 Pinter89 Probabilistic Inequalites
6/21
224 J. Pint4r
Then
[62 + (u - / l ) (a - / a ) ] In {1
+ ( u - I ~ ) (a - t i ) / 6 z } + (u - U ) ( u - a )
l n { 1 - ( a - l i ) / ( u - t ~ ) } ~ > [ ( u - / a ) e + 6 2 ] l n ( 1 / s ) ( / l < a K u )
2.13)
implies the rela tion P(~ >~ a) ~< ~.
P r o o f : Accord ing to Theorem 3 o f Hoe ffd ing (1963) , fo r indepe nden t r.v .' s ~ i , E~ i = O ,
~ i < ~ b , i = 1 , 2 , . . . , n
and fo r 0 < t < b the inequa l i ty
P(~ >t t) ~ {(1 +
b t / 6 2 ) - ( l + b t / ~ 2 ) ~ 2 / ( ~ 2 + ~ 2 ) ( 1 - t / b ) - ( 1 - t / b ) a 2 / ( b 2 + ~ 2 ) } n
(2 .14)
holds , w here (as earl ier ~ = ( ~
~ i ) /n . T a k i n g n = l , f i r s t t r a n s f o r m t h e r . v . ~ : = ~ -
to y ield E~ = 0 . ( th is impl ies tha t on the r ight-hand s ide the up per bo un d b is to be
rep laced by u - / l . ) Now , it i s eas ily seen tha t (2 .13) and (2 .14) lead to the inequal i ty
(1.1).
R e m a r k 2 . 4 :
As shown by Hoeffd ing (1963) , inequal i ty (2 .14 ) i s the best (cf . Rem ark
2 .3 ) tha t can be ob ta ined
a p p l y i n g B e r n s t e in ' s t e c h n i q u e -
an d k n o w i n g
o n l y
the
m e a n , v a r i a n c e
and an
u p p e r b o u n d
of ~.
(Note here tha t - as was po in ted ou t by one o f the Referees - (2 .6 ) i s
t h e s h a r p e s t
bou nd , i f / l ~< a < u ; fu r the r ,
- l)/ (a - l)
-
8/10/2019 Pinter89 Probabilistic Inequalites
7/21
Deterministic Ap proximations of Probabil ity Inequalit ies 225
d e t e r m i n i s t i c r e l a t i o n s w h i c h i m p l y t h e p r o b a b i l i t y i n e q u a l i t y
P ~ > a ) = P ~ i~ > n a )~ < ~
( 3 . 1 )
F i r s t , t h e o n e - s i d e d C h e b y s h e v - i n e q u a l i t y a n d t h e t w o - s i d e d C h e b y s h e v - i n e q u a l i t y
a r e a p p l i e d f o r
a r b i t r a r y
( in c l ud i n g c o r r e l a te d a n d / o r u n b o u n d e d ) s u m m a n d s ; th i s w i ll
b e f o l l o w e d b y a p p li c a t io n s o f th e B e r n s t e in - ty p e e x p o n e n t i a l e s t i m a t e s f o r
i n d e p e n -
d e n t
a n d
b o u n d e d
( f r o m b o t h s i d e o r f r o m a b o v e ) s u m m a n d s .
A c c o r d i n g t o P r o p o s i t i o n 2 . 1 ( s e e (2 . 6 ) ), /1 +
6 X / ~ / a - 1 a ) ~ < a . I f t h e ~ i s a r e i n d e p e n d e n t , t h e n o b v i o u s l y 6 2 = 2 ; 6 ~ / n Z ;
t h e r e f o r e i n t h is c a s e ( 2 . 6 ) r e a d s a s
J l )
~i + n 6 ] 1 / a - 1 ) < . n a
( 3 . 2 )
i m p l y i n g ( 3 . 1 ) .
F u r t h e r , i n t h e c a s e o f s y m m e t r i c a l l y d i s t r i b u t e d s u m m a n d s ~ i , t h e B i e n a y m G
C h e b y s h e v i n e q u a l i t y y i e ld s
P ( I ( - / J [ ~> t ) = 2P(~ ~>/a + t ) ~~ O ,
i .e. ~ /l i + 6/2 /( 2 a ) ~ ~ + t ) < . E e x p { h [ ~ 1 ~ i - n ( l a + t ) ] I ( h > O )
( 3 . 4 )
N o w , i f t h e s u m m a n d s ~ i, i = 1 . . . . . n a r e i n d e p e n d e n t , t h e n
e x p f I t : , e x (
T h e r e f o r e i t r e m a i n s t o o b t a i n
s e p a r a t e
b o u n d s f o r th e e x p e c t e d v a l u e f a c t o r s in ( 3 . 5 )
a n d t h e n - w i t h t h e p u r p o s e o f s h a r p e n i n g t h e re s u lt - - t o m i n i m i z e th e a g g r e g a te d
r i g h t- h a n d s id e b o u n d w i t h r e s p e c t t o h . T h i s m e t h o d is a d v a n t a g e o u s , a s o p p o s e d t o
t h e s t r a ig h t f o r w a r d a p p r o a c h o f t r e a ti n g t h e s u m r .v . ~ i n t h e m a n n e r o f H o e f f d i n g
( 1 9 6 3 ) , w h e n t h e r . v . s ~ i a r e n o t i d e n t i c a l l y d i s t r i b u t e d a n d t h e i r r a n g e s , v a ri a n c e s ar e
d i f f e r e n t ( o r e v e n t h e a v a i l a b l e i n f o r m a t i o n m a y v a r y s e p a r a t e l y f o r t h e i n d i v i d u a l
r . v . s ~ i ) -
F o r e x a m p l e , l e t u s a s s u m e t h a t l i < ~ ~ i ~ U i , i = 1 . . . . n ( - - ~ < I < u i < ~ ) . B y t h e
c o n v e x i t y o f t h e f u n c t i o n e x p
( h x ) ,
fo r a r b i t ra ry l , x , u , l ~< x ~< u w e h ave
e x p
( h x ) < .
[ (u -
x ) / ( u -
l ) l e x p
( h l ) +
[ (x -
l ) / ( u -
l ) l e x p
( h u ) ( h > O )
Th er e f o r e w e have f o r t he r .v . s ~ i, i = 1 , . . . , n t he r e spec t i ve i nequa l i t ie s
E e x p
( h ~ i ) < . [ ( u i - i ) / ( u i - l i ) ]
e x p
( h l i )
+ [(/.ti - l i ) / ( u i - l i ) ] e x p ( h u i ) ( 3 . 6 )
S u b s t i t u ti n g ( 3 . 6 ) i n t o ( 3 . 5 ) w e o b t a i n
E e x p h ~ i - n 0 ~ + t ) = e x p
{ - h n ( l ~ +
t ) } I I e x p
( h ~ i )
I
~q
~ < e x p
{ - h n ( l a +
t )} 1 ] { [ ( u i - l a i ) / ( u
i - l i ]
e x p
( h l i )
?
+ [ ( U i - l i ) / ( U i - l i ) ] e x p ( h u i ) ) = e x p { - h n ( + t)} 1-I R i ( h )
-
8/10/2019 Pinter89 Probabilistic Inequalites
9/21
Determin ist ic Ap proxim at ions of Proba bi l i ty Inequa l it ies 227
D e n o t i n g t h e l a s t e x p r e s s i o n o b t a i n e d b y
R ( h ) = R ( h , t , I J i , l i ,
b l i , i = 1 . . . . . n
w e h a v e
W e c a n s u m m a r i z e t h e a b o v e a r g u m e n t a t i o n in th e f o r m o f
P r o p o s i t i o n 3 . 1 : I f t h e r . v . s ~ i, i = 1 , . . . , n h a v e f i n i t e m e a n s / a i a n d r a n g e s [ l i , u i ] , t h e n
t h e i n e q u a l i t y
R ( h )
= e x p
{ - h n a } F I
1
{ [ ( u i -
U i ) / ( u i - l i ) ]
e x p
( h l i )
+ [(Ui - l i ) / ( u i - l i ) ] e x p ( h u i ) } < ~
( 3 . 7 )
w i t h a r b i t r a r y h > 0 im p l i e s t h e r e l a ti o n P ~ i > ~ n a < ~ ~ .
I n p a r t i c u l a r , t h e b e s t b o u n d o b t a i n a b l e b y B e r n s t e i n s m e t h o d i s g i v en b y R ( h * ) < ~ a ,
w h e r e h * s o l v e s R ( h * ) = r a i n R ( h ) .
h>O
W e sh a ll s h o r t l y d i sc u s s a ls o t h e c a s e , w h e n t h e a p p r o x i m a t i o n i s b / ls e d o n t h e
k n o w l e d g e o f m e a n , v a r ia n c e a n d u p p e r b o u n d v a l u e s. I n B e n n e t t ( 1 9 6 2 ) t h e i n -
e q u a l i t y
E e x p ( h ~ ) ~ < [ b 2 / ( b 2 + 6 2 ) ] e x p ( - g 2 h / b ) + [ ~ 2 / ( b 2 6 2 ) ] e x p ( h b ) ( 3 . 8 )
is p r o v e d f o r a r b i t r a r y h > 0 , w h e r e b y s u p p o s i t i o n E ~ = 0 , E ~ 2 = 6 2 a n d ~ ~< b . A p p l y -
i n g t h i s r e l a t i o n f o r t h e r . v . s
~ i - f J i , i = 1 , . . . , n ( ~ i - l a i < ~ u i - I a i )
w e o b t a i n
E e x p
{ h ( ~ i - l a i ) ) < ~ ( ( u i - i J i ) 2 / [ ( u i - U i )
2 + 6 2 ] } e x p
{ - h 6 ] / ( u i - i ) }
+ ~ 6 ] / [ ( t t i _ / / i )2 + 6 ] ] } e x p { h ( u i - / . t i ) } = S i ( h )
-
8/10/2019 Pinter89 Probabilistic Inequalites
10/21
228 J. Pint6r
T h e r e f o r e i n t h i s c a s e w e o b t a i n t h e r e l a t i o n s
P ~i >~ ~ #i + n t
-
8/10/2019 Pinter89 Probabilistic Inequalites
11/21
Determin ist ic Ap prox im at ions o f Probab i l i ty Inequa li ties 229
Remark 3.3. F o l l o w i n g H o e f f d i n g ( 1 9 6 3 ) , t h e c as e o f dependent s u m m a n d s is e a s il y
d e d u c i b l e t o t h e p r e v i o u s a n a l y s i s . A s s u m e t h a t t h e r . v . T i s a s u m o f t h e f o r m
N N
T= ~ p iT] p]>~O ~, p l= l
( 3 . 1 0 )
1 1
w h e r e e a c h
T]
i s a s u m o f i n d e p e n d e n t r .v . 's n o t e t h a t t h e T i 's n e e d
not
t o b e i n d e p e n -
d e n t . ( E v i d e n t l y , t h e c a s e o f d e p e n d e n t s u m m a n d r . v .' s c a n b e i n t e r p r e t e d a s a s p e c ia l
c a se o f ( 3 . 1 0 ) . ) B y B e r n s t e i n 's i n e q u a l i t y w e h a v e
P (T > / t )< ~ e x p ( -h t )E e x p (h T ) (h > 0 )
F u r t h e r , b y th e c o n v e x i t y o f t h e e x p o n e n t i a l f u n c t i o n o n e c a n a p p l y J e n s e n ' s i n e q u a li -
t y f o r a r b i t r a r y a r g u m e n t s t = N pit] t h i s g iv e s
N )N
x p ( h t ) = e x p h ~ pl ti ~ 1 p j e x p ( h tj )
T h e r e f o r e t a k i n g th e e x p e c t a t i o n o f e x p { h ( T - t ) } w e o b t a i n
N N
P ( T > / t )< . X p i e e x p { h ( T j - t ) } = s p /Q /
H e r e t h e s u m m a n d f a c to r s Q j - d e f i n e d b y s u m s o f independent r . v . ' s c a n b e
b o u n d e d s e p a r a t e l y , a p p l y i n g P r o p o s i t i o n s 3 .1 a n d 3 .2 .
N o t e f i n a ll y t h a t t h e f r a m e w o r k o f S e c t i o n 3 o b v i o u s l y a p p l i e s t o a ll l i n ea r c o m -
n
b i n a t i o n s ~ = 23
e i ~ i
w h e r e t h e ei 's a r e g i v e n r e a l c o e f f i c i e n t s .
S o m e P o s si b le A p p l ic a t i o n s
T h e e x a m p l e s s h o w n b e l o w s e rv e p r i m a r i l y f o r th e p u r p o s e o f i l lu s t r at i n g t h e p o t e n -
t ia ls o f t h e s u g g e s t e d a p p r o a c h : h e n c e , d e t ai ls u n n e c e s s a r y i n t h e p r e s e n t c o n t e x t w i ll
b e o m i t t e d o r o n l y s h o r t l y r e f e r re d t o .
-
8/10/2019 Pinter89 Probabilistic Inequalites
12/21
230 J . Pint6r
4 .1 G e n e r a l i ze d ( S t o c h a s ti c ) K n a p s a c k P r o b l e m
T h e s o- c al le d k n a p s a c k p r o b l e m ( K P ) is a ( li n ea r ) i n te g e r p r o g r a m m i n g p r o b l e m
w i t h a s i n g l e i n e q u a l i t y c o n s t r a i n t :
m a x c T x c C R~ )
d T x < . b d E R ~ , b > O )
( 4 . 1 )
x T = x I , . . . , X n ) x i = 0 o r l , i = 1 . . . . . n
T h e c o n c e p t w h i c h l e a ds t o th e m o d e l ( 4 . 1 ) c a n b e v e r b a l ly s t a t e d e .g . i n t h e f o l l o w i n g
w a y : f i n d a p o r t f o l i o o f p o s si b l e y e s / n o a c t i o n s s u c h t h a t i t s s u m m e d b e n e f i t s ( p o s i ti v e
i m p a c t s ) a r e m a x i m a l , w h i le i t s t o t a l c o s t s ( r e so u r c e d e m a n d s ) d o n o t e x c e e d a p r e f i x e d
u p p e r b o u n d . A m o n g t h e m a n y a p p l i c a ti o n s o f t h is m o d e l , w e m e n t i o n o n l y t he r e c e n t
w o r k o f L o o t s m a , M e i s n e r a n d S c h e l l e m a n s ( 1 9 8 6 ) , in w h i c h t h e m u l t i o b j e c t i v e p r o b -
l e m o f n a t i o n a l e n e r g y r e s e a r c h a n d d e v e l o p m e n t p l a n n i n g is r e d u c e d - v i a w e i g h i n g
s c a l a r i z a t i o n o f t h e o b j e c t i v e s - t o th e f o r m ( 4 . 1 ) .
N o w , i f - i n s t e a d o f a c c e p t i n g f i x e d v a l u es e i > 0 - w e a s s u m e t h a t t h e b e n e f i t s
a r e u n c e r t a i n t o s o m e e x t e n t , t h e n t h e y c a n b e m o d e l l e d b y i n t r o d u c i n g r .v . 's rT i,
i = 1 , . . . , n. I n t h i s c a s e i t m i g h t b e r e a s o n a b l e t o s e l e c t p o r t f o l i o s w h i c h assure a t l eas t
s o m e f i x e d t o t a l r e t u r n i n a p r o b a b i l i s t ie s e n se . T h i s i d e a m a y l e a d t o t h e f o l l o w i n g
s t o c h a s t i c e x t e n s i o n o f t h e K P (4 . 1 ) :
m a x P r f x >~ a ) ( a > 0 e x t e r n a l l y g i v en m o d e l p a r a m e t e r ) ( 4 . 2 )
dTx ~b
w i t h t h e i n t e g r a l i t y a s s u m p t i o n s x i = 0 o r 1 . I n t h e f o l l o w i n g d e t e r m i n i s t i c a p p r o x i m a -
t i o n s t o t h e o b j e c t i v e f u n c t i o n o f ( 4 . 2 ) w il l b e d e ri v e d , w h e n a d d i t i o n a l i n f o r m a t i o n
c o n c e r n i n g t h e r . v . ' s r/i i s a v a i l a b l e . ( N o t e t h a t - a s t h e r e t u r n s o f t e n h a v e s t r o n g l y
s k e w e d p . d .f . ' s - t h e a p p r o x i m a t i o n i n d u c e d b y th e c e n tr a l l im i t t h e o r e m m i g h t be
r a t h e r c r u d e f o r s m a l l v a l e s o f n , w h i le t h e a n a l y t i c a l e x p r e s s i o n o f t h e p . d . f, o f r / r x =
~ , p i x i - e v e n f o r f i x e d v a l u e s x i - is u n k n o w n i n m a n y c a se s .)
1
B y t h e r e s u l ts p r e s e n t e d e a r li e r, w e k n o w s e v e ra l i n e q u a l i ti e s o f th e f o r m
P ~ > i a )
-
8/10/2019 Pinter89 Probabilistic Inequalites
13/21
Deterministic Approxim ations of Probability Inequalities 231
{ P B }
b e i n g a j o i n t s y m b o l i c n o t a t i o n fo r t h e p ro b ab i l i t y b o u n d s o b t a i n ed . Hen ce , i t
is r ea s o n ab le t o fo l l o w t h e t w o -s t ep p ro ced u re b e l o w:
i) Select f i rs t the a d a p t i v e p a ram e t e r s o f { P B ) in such a way th a t { P B } is m i n i m i z e d .
This ac t ion makes the bo und sha rpes t , imp ly ing tha t P(~ ~>a) and { P B } are
as close as pos s ible in the g iven fram ew ork . (Du ring th is s tep the x i s are regarded
as ex o g en o u s l y f i x ed p a ram e t e r s , t h e re fo re t h e b o u n d i n g p ro ced u re s h o u l d b e
val id for their
al l
feas ible set t ings . )
i i) R e p la c e t h e p r o b l e m o f m a x i m i z in g P ( ~ / > a ) b y
m a x i m i z i n g
{PB}: th is act ion is
b as ed o n t h e
q u a n t i t a t i v e i n f o r m a t i o n
ab o u t t h e co m p o n e n t s ~ i o f ~ ( t hi s i n fo rm a-
t i o n d e t e rm i n es an ap p r o x i m a t e l y b e s t s e t ti n g o f t h e
d e c i s io n v a r ia b le s
{xi}) .
I f thi s c onc ep t i s app l ied to the r .v . ~ = r S x = 2; 7 7 ix i , then the ob jec t ive func t io n in
I
(4 . 2 ) w i ll b e r ep l aced ( s y m b o l i ca ll y ) b y d e t e rm i n i s ti c ex p re s si o n s o f t h e fo rm
m a x
{ P B ( x ) }
(4 .4 )
T h e c o n c r e t e f o r m o f
{ P B ( x ) }
d ep en d s , o f co u r s e , o n t h e q u an t i t a t i v e i n fo rm a t i o n
ab ou t the r /i 's . Be low we shal l g ive exam ples o f the ob jec t ive func t ion (4 .4 ) , app ly ing
some resu l ts o f Sec t ion 3 .
Assume f i r s t tha t fo r a l l r .v . 's rl i we kn ow (o r can es t im ate) the i r low er /u ppe r
b o u n d s an d m ean . B y P ro p o s i t io n 3 . 1 , fo r an a rb i t r a ry s u b s e t o f in d ice s I C { 1 . . . . . n }
and in dep end en t r .v . 's ~ i i E I wi th kn ow n range and me an , we have
P ~ ~ i >~ a ) ~< ex p { - h a } F I { [ u i - t ~ i ) / u i - l i ) e x p h l i )
i E l i ~ I
+ [(/.zi -
l i ) / (u i - l i ) ]
e x p
( h u i ) } .
Th erefo re fo r a ll index-se t s I ch arac te r ized by
x i = 1 i E I ,
we have
l n P ~ i x i > ~ a = l n P ( G ~ i > ~ a ) (4 .5 )
iE
< ~ - h a
+ 2 ; I n { [ u / - U i ) / u i - l i ) ] ex p
( h l i )
i ~ l
+ [(/ai -
l i ) / (u i - l i )]
e x p
( h u i ) }
-
8/10/2019 Pinter89 Probabilistic Inequalites
14/21
232 J. Pint6r
l l
= - h a + ~ i n { [ ( u i - U i ) / ( u i - l i ) ] e x p ( h l i )
1
+ [ ( # i - [ i ) / ( u i - l i ) ] e x p ( h u i ) } x i
K e e p i n g n o w t h e p a r a m e t e r v a l ue s a a n d h > 0 f i x e d ( i .e . a l r e a d y s e le c t e d ) , f r o m ( 4 . 5 )
w e c a n c o n c lu d e t h a t m a x i m i z i n g t h e p r o b a b i l i t y b o u n d
{ P B ( x ) }
i n ( 4 . 4 ) l e a d s t o t h e
m a x i m i z a t i o n o f t h e s u m i n t h e la s t e x p r e s s i o n o f ( 4 .5 ) . I n o t h e r w o r d s , t h e s t o c h a s t i c
K P ( 4 . 2 ) is a p p r o x i m a t e d b y t h e f o l l o w i n g d e t e r m i n i s t i c K P :
n
m a x Y , c } 1 )x i
d T x ~ b
( 4 . 6 )
w h e r e
c} 1) = in { [ ( u i -
t J i ) / ( u i - l i) ]
e x p
( h l i ) +
[ (g i -
l i ) / ( u i
- l i ) ]
e x p
h u i ) }
a n d
x i = O o r 1, i = 1 , . . . , n .
N o t e t h a t t h is d e t e r m i n i s t ic r e f o r m u l a t i o n o f p r o b l e m ( 4 . 2 ) i s p e r f e c t , w h e n i n ( 3 . 6 )
e q u a l i t y s t a n d s ( i. e. a l l r . v . 's r/i h a v e t w o - p o i n t d i s c r e t e d i s t r i b u t i o n s w i t h
P ( ~ i = l i ) =
( u i - # i ) / ( u i - l i ) a n d P ( ~ i = u i ) = ( g i - l i ) / ( u i - l i ) ) . N o t e a l so t h a t a l t h o u g h t h e e x o g e n -
o u s l y g i ve n a s p i r a t io n l e v e l a d o e s n o t o c c u r e x p l i c i t l y i n t h e o b j e c t iv e f u n c t i o n o f
p r o b l e m ( 4 .6 ) , t h e p a r a m e t e r v al u e h ( w h i c h m i n i m i z e s t h e b o u n d ) d e p e n d s o n a ;
h e n c e , t h e c o e f f i c i e n t s c ~ 1) a r e im p l i c i t f u n c t i o n s o f a . W e e m p h a s i z e t h a t h h a s t o
b e f i x e d i n d e p e n d e n t l y o f t h e s e l e c t i o n o f th e i n d e x - s e t I ( x i = 1 , i E / ) , f o r t h e s a k e o f
c o m p u t a t i o n a l s i m p l i c i t y : t h e r e f o r e I = I o = ( 1 . . . . . n } se e m s t o b e t h e m o s t n a t u r a l
c h o i s e . L e t u s n o t e f i n a l l y t h a t i n t h e c a s e o f u i = l i = g i = c i , i = 1 , . . . , n ( i . e . w h e n
o u r r .v . 's r e d u c e t o e x a c t p o i n t e s t i m a t e s
c i
o f t h e b e n e f i t s ) w e h a v e c~ l ) =
h c i ( h >
0 ) :
t h is w a y , t h e a p p r o x i m a t i n g s t o c h a s t i c K P f o r m u l a t i o n ( 4 . 6 ) i n c l u d e s a s a s p e c ia l c a se
t h e i n i t i a l , d e t e r m i n i s t i c K P .
A n a l o g o u s c o m m e n t s a r e v a li d a l so i n t h e c a s e , w h e n t h e m e a n /~ g , u p p e r - b o u n d u i
a n d v a r i an c e 6 i p a r a m e t e r s a r e k n o w n / e s t i m a t e d f o r a ll ~i 's . W i t h o u t r e p e a t i n g t h e
p r e v io u s a r g u m e n t a t i o n , o n e c a n s ee t h a t t h e s t o ch a s t ic K P ( 4 .2 ) ca n b e a p p r o x i m a t e d
b y
-
8/10/2019 Pinter89 Probabilistic Inequalites
15/21
Determin ist ic Ap prox imat io ns o f Proba b i l i ty Inequa li ties 233
i
m a x O } 2 ) X i
d T x ~ b
( 4 . 7 )
w h e r e
e} 2) : In I u i - I R i ) 2 / [ u i - / 2 i ) z + 6 2 1 e x p [ - h 6 ~ / u i - /~ i )1
+ 6 2 / [ u i - I Ji ) 2 + 6 2
] ) e x p
[ h u i - / l i ) ]
i +
ht2 i
a n d
x i = O
o r l , i = l , . . . , n .
A s e a rl i e r , t h e a s p i r a t i o n l e v e l a i s i m p l i c i t l y i n c l u d e d v i a t h e p a r a m e t e r h . B e si d e s, i t
i s e a s y t o s e e a g a i n t h a t
c ~ 2 ) - ~ h l J i , i = 1 , . . . , n ,
w h e n
u i - ~ l~ i
a n d 6 i ~ 0 .
C o n c l u d i n g t h i s e x a m p l e w e n o t e t h a t - a s i t is w e l l k n o w n - ( s u b ) o p t i m a l s o lu -
t i o n s t o t h e d e t e r m i n i st i c K P ca n b e e f f ic i e n c tl y g e n e r a t e d , f o r m i n g t h e m o n o t o n o u s l y
d e c r e a s in g o r d e r o f b e n e f i t / c o s t r a t io s c i / d i } . T h u s , a s i m i la r p r o c e d u r e c a n b e a p p l i e d
t o t h e d e t e r m i n i s t i c a p p r o x i m a t i n g p r o b l e m s (4 . 6 ) , ( 4 . 7 ) : t h i s m a y b e h e l p f u l e .g . i n
e x p l o r i n g t h e v a r i a t i o n s o f t h e p r o j e c t r a n k s a s a f u n c t i o n o f th e t a r g e t le v e l a .
4.2 Chance C onstrained Linear Programming
T h e li n ea r p r o g r a m m i n g p r o b l e m ( e.g . f o r f in d i n g th e m i n i m u m - c o s t p r o d u c t i o n p l a n
w h i c h s a t is f ie s d i f f e r e n t t y p e s o f d e m a n d s ) c a n b e s t a t e d a s
m i n cTx
a T x > / b i i = 1 , . . . , m
x E R ~ _ , a i , c E R n )
4 .8 )
T h e f o r m u l a t i o n ( 4 . 8 ) m a y h a v e d i f f e r e n t e x te n s i o n s , w h e n p o s s ib l e u n c e r t a i n t i e s ,
c o n c e r n i n g t h e c o s t - c o e f f i c i e n t s
c ] ] = 1 , . . . , n ,
p r o d u c t i o n t e c h n o l o g y c o e f f i c i e n t s
-
8/10/2019 Pinter89 Probabilistic Inequalites
16/21
234 J. Pint6r
a q a i = ( a n . . . . , a i n ) , i = 1 . . . . . m ) and demand leve l s b i i = 1 . . . . . m , are a l so t aken
in to cons idera t ion . A poss ib le s tochas t ic re fo rmula t ion o f (4 .8 ) has the fo l lowing
f o r m :
m i n E
E c j ) x j
P ~ a g x j > b i ) > t 1 - e i i = l , . . . , m
0 < e i < 1 p a ram e t e r s )
(4 .9 )
Acco rd i n g t o (4 . 9 ) , t h e ex p ec t ed (p ro d u c t i o n ) co s t s a r e t o b e m i n i m i zed , wh i l e t h e
sa t i s fac tion o f the (dem and ) cons t ra in t s is to be assu red wi th respec t ive , su f f ic ien t ly
h igh p robab i l i t i es . No te tha t (4 .9 ) i s n o t t h e u n i q u e o r b e s t s t o ch as t i c v e r si o n
of (4 .8 ) : i t s use can be reasonab le e .g . in such s i tua t ions , when the vec to rs a i can be
cons idere d as (ap pro x im ate ly ) indep ende n t , wh i le there a re s ign i fican t d i f fe rences be-
t ween t h e i m p o r t an ces o f m ee t i n g t h e d em an d s b i . Th erefo re (4 .9 ) can be a pp l ied e .g .
in mo del l ing cer ta in a l loca t ion /d i s t r ibu t ion p rob lem s , when exp l ic i t p r io r i ty o rde r
be tween the d i f fe ren t demands i s to be cons idered .
Assuming tha t the mean va lues E c ] ) a re k n o wn (o r p ro p e r l y e s t i m a t ed ) , t h e o b -
jec t ive func t ion o f (4 .9 ) i s aga in l inear . Below we p resen t exp l ic i t approx imat ions o f
the separa te chance -cons t ra in t s , on the bas is o f the p rev ious es t imates . (Note th a t
ear l ie r S inha 1963 app l ied the Bienaym &C hebyshev inequ al i ty wi th the same ob jec t ive ,
see also Wets 1983.)
Def ine
~ i o = b i i = l . . . . . m , ~ i] = - a i i j = l , . . . , n , i = l , . . . , m ,
fu r t h e r o n , l e t
x o = l . T h e n w e h a ve t h e t r a n s c r i b e d p r o b a b i l i s t i c c o n s t r a i n t s P ( o ~ ~ i l x j < < , O ) > l - e i ,
i.e.
P ~ o ~ i l x i > ~ O ) < ~ e i i = 1 . . . . . rn ( 4 . 1 0 )
(by assu mp t ion , a l l occur ing r.v .' s have co n t inu ous p .d . f .' s ) .
By Propo s i t ion 2 .1 we know tha t (app ly ing the one-s ided Chebyshev- ineq ual i ty )
the re la t ion
/1 + 8 X /1/a - 1 ~ a im plie s P(~ >~ a) ~