basics of probability in statistical simulation and stochastic programming
DESCRIPTION
Basics of probability in statistical simulation and stochastic programming. Lecture 2. Leonidas Sakalauskas Institute of Mathematics and Informatics Vilnius, Lithuania EURO Working Group on Continuous Optimization. Content. Random variables and random functions Law of Large numbers - PowerPoint PPT PresentationTRANSCRIPT
Basics of probability in statistical simulation and stochastic programming
Leonidas SakalauskasInstitute of Mathematics and InformaticsVilnius, LithuaniaEURO Working Group on Continuous Optimization
Lecture 2
Content
Random variables and random functions
Law of Large numbers Central Limit Theorem Computer simulation of random
numbers Estimation of multivariate integrals
by the Monte-Carlo method
Simple remark
Probability theory displays the library of mathematical probabilistic models
Statistics gives us the manual how to choose the probabilistic model coherent with collected data
Statistical simulation (Monte-Carlo method) gives us knowledge how to simulate random environment by computer
Random variable
Random variable is described by
Set of support Probability measure
Probability measure is described by distribution function:
)(Pr)( xXobxF
)(XSUPP
Probabilistic measure
Probabilistic measure has three components:
Continuous;Discrete (integer);Singular.
Continuous r.v.
Continuous r.v. is described by probability density function )(xf
Thus:
x
dyyfxF )()(
Continuous variable
If probability measure is absolutely continuous, the expected value of random function:
dxxpxfXEf )()()(
Discrete variable
Discrete r.v. is described by mass probabilities:
n
n
ppp
xxx
,...,,
,...,,
21
21
Discrete variable
If probability measure is discrete, the expected value of random function is sum or series:
n
iii pxfXEf
1
)()(
Singular variable
Singular r.v. probabilistic measure is concentrated on the set having zero Borel measure (say, Kantor set).
Law of Large Numbers (Chebyshev, Kolmogorov)
,lim 1 zN
zN
i i
N
here are independent copies of r. v. ,
Ez
Nzzz ,...,, 21
What did we learn ?
N
N
i i 1
is the sample of copies of r.v. , distributed with the density .
Nzzz ,...,, 21
dzzpzxf )(),(The integral
is approximated by the sampling average
,,...,1),,( Njzxf jj if the sample size N is large, here
)(zp
Central limit theorem (Gauss, Lindeberg, ...)
),(/
lim xxN
xP N
N
,2
1)( 2
2
x y
dyex
here
,EX 222 )( XEXD,1
N
x
x
N
ii
N
Beri-Essen theorem
N
XExxFN
x
3
3
41.0)()(sup
xxobxF NN Pr)(where
What did we learn ?
According to the LLN:
,1
1
N
iixN
x,
)(1
2
2
N
xxN
iNi
N
xx
EXXE
N
iNi
1
3
3
Thus, apply CLT to evaluate the statistical error of approximation and its validity.
Example
Let some event occurred n times repeating N independent experiments.
Then confidence interval of probability of event :
N
ppp
N
ppp
)1(96.1,
)1(96.1
here ,N
np (1,96 – 0,975 quantile of normal distribution,
confidence interval – 5% )
6)1( ppNIf the Beri-Esseen condition is valid: !!!
Statistical integrating …
b
a
dxxfI )( ???
Main idea – to use the gaming of a large number of random events
Statistical integration
dxxpxfXEf )()()(
,)(
1
N
xfNi i )(pxi
Statistical simulation and Monte-Carlo method
x
dzzpzxfxF min)(),()(
,min),(
1
x
N
i i
N
zxf )(pzi
(Shapiro, (1985), etc)
Simulation of random variables
There is a lot of techniques and methods to simulate r.v. Let r.v. be uniformly distributed in the interval (0,1]
Then, the random variable , where ,
)(UFU
is distributed with the cumulative distribution function )(F
))sin(cos()( xaxxxf
0
))sin(cos()( dxexaxxaF x
N=100, 1000
Wrap-Up and conclusions
o the expectations of random functions, defined by the multivariate integrals, can be approximated by sampling averages according to the LLN, if the sample size is sufficiently large;
o the CLT can be applied to evaluate the reliability and statistical error of this approximation