Monte Carlo Simulation

Fawaz hrahshehDr. A.

obeidatDepartment of

physics just

History

What is Monte Carlo (MC) method ?

The Monte Carlo method :is a numerical method for statistical simulation which utilizes sequences

of random numbers to perform the simulation

What the meaning of MC simulation?

●MC simulation is a versatile tool to analyse and evaluate complex measurements

● Constructing a model of a system.

●Experimenting with the model to draw inferences of the system’s behavior

A simulation model

Decision and

uncontrollable

variables

Simulati

on

model

Measures of

performance

or behaviour of

the system

Inputs outputs

A simulation model cont..• Model inputs capture the environment of the

problem

• The simulation model

– Conceptual model: set of assumptions that define

the system

– Computer code: the implementation of the

conceptual model

• Outputs describe the aspects of system behaviour

that we are interested in

Random numbers

• Uniform Random numbers or pseudo-random numbers (PRN) are essentially independent random variables uniformly Distributed over the unit interval (0,1).

●The PRNs are good if they are uniformly

distributed, statistically independent and

reproducible.

Linear congruential generator

• Generating a random sequence of numbers {X1,X2,…….,Xk} of length M over the interval [0,M-1]

Xi=mod(AXi-1+C,M) R=Xi/M♠♠♠ mod(b,M)=b-int(b/M)*M● Starting value X0 is called “seed”●M,A and C are nonnegative integers knownModulus, multiplier and increment, respectively●M is must be prime number(2³¹-1,2-1,…..)

program random_num implicit none

integer,parameter::m=2**21,a=17,c=43 integer::i,n,s real(8)::R,Xo

read*,n !n is the number of random points

Xo=27.0d0!------------------------ do i=1,n s=A*Xo+c Xo=mod(s,m)

R=Xo/m write(*,*) R

end do !------------------------ end program random_num

Fortran program

Classic Example

44

2

2

r

r

circleofarea

squareofarea

Find the value of ?

Use the reject and accept methodOr hit and miss method

The area of square=(2r)²

The area of circle = r²

squareofarea

circleofarea

*4

Cont....

dotsofnumbertotal

circleinsidedotsof

squareofarea

circleofarea

...

...#.

..

..

Hit and miss algorithm♣ Generate two sequences of N of PRN :: Ri,,Rj

♣ Xi=-1+2R i

♣ Yj=-1+2R j

♣ Start from s=zero♣ If (X²+Y²<1) s=s+1

♣ # of dots inside circle=s♣ total number of dots=N

NS /*4

PROGRAM Pi implicit none integer,parameter::mm=(2**17)-1,aa=(2**7)-1

REAL(8)::X, Y, Pi Real(8),dimension(:),allocatable::R

INTEGER::I,N Pi = 0.0d0 READ*, N allocate(R(n)) DO I = 1,N CALL RANDOM_NUMBER(R(n)) CALL RANDOM_NUMBER(R(n+1))

X =-1.0d0+2.0d0*R(n) Y=-1.0d0+2.0d0*R(N+1)

IF (X*X+Y*Y<1.0d0) Pi=Pi+1.0d0 END DO Pi=4*Pi/N PRINT*,Pi END program pi

Fortran program

Monte Carlo Integration

♥ Hit and miss method

♥ Sample mean method

♥ importance sampled method

Hit and Miss method

♦Generate two sequence of N of PRN (Ri,R j) i& j=1,2,….,N

b

a

dxxfI )( Rba ,

♦0 ≤f(x) ≤Ymax ,for X Є (a,b)♦ Xi=a+Ri (b-a)

♦ Yi=Ymax R j

♦ start from s=0

♦ if Yj<f(x) s=s+1

♦ I=Ymax(b-a) S/N

program Hit_miss implicit none real(8),dimension(:),allocatable::Rreal(8)::X,Y,m,a,b,integinteger::i,N,sread*,a !the lower value of the intervalread*,b !the upper value of the intervalM=f(b)if (dabs(f(a))>dabs(f(b))) M=f(a)read*,N !the number of random numbersallocate(R(n))s=0do i=1,N call random_number(R(n))call random_number(R(n+1))X=a+R(n)*(b-a)Y=M*R(n+1)if (y<f(x)) s=s+1end doINTEG=M*(b-a)*s/Nprint*,integcontainsreal(8) function F(X)real(8),intent(in)::XF=2.0*X+1.0end function Fend program Hit_miss

Cont….If there is a part of the function under X-axis

))((

)(

.

..

abRM

dxxf

areaTotal

curveunderarea

N

sb

a

b

a N

sabRMdxxf ))(()(

This must now be corrected for the fact that the line y = -R has been used as the baseline for the integral instead of the line

y = 0. This is accomplished by subtracting the rectangular area R(b-a).

b

a

abRN

sabRMdxxf )())(()(then

Types of distribution

Gaussian or normal distributionUniform distribution

Sample Mean method

Rewrite b

a

dxxfI )( By a

a

dxxxhI )()(

b

a

dxx 1)(0)( xWhere is

p.d.f

If x1,x2,x3,…..,xN are i,i,d uniformly distributed on [a,b],then fabdxxfI

b

a

)()(

N

iixf

Nf

1

)(1,

Theorem….)()()( xxfxh

From the theorem choose and Cont…

abx

1

)( )()()( xfabxh

Then an estimate of I is

N

iixf

N

abI

1

)()(ˆ

f

a b

F(x)

You can calculate the value of error from the variance

)ˆvar(Ierror

22)var( fff

)var()(

)ˆvar(2

fN

abI

x

N

ii

N

ii

xfN

xfN 1

2

1

2 )](1[)(

1

Sample Mean MC algorithm♠ Generate sequence of N of PRN : Ri

♠Compute Xi=a+Ri (b-a)♠ compute f(Xi) , i=1,2,3,….,N

N

iixf

N

abI

1

)()(ˆ♠ use

♠♠♠ note:: if f(x) is not square integrable ,then the MCEstimate Î will still converge to the true value, but

The error estimate becomes unreliable.

program Sampled_Meanimplicit nonereal(8),dimension(:),allocatable::Rreal(8)::a,b,sum,integ,Xinteger::i,Nread*,a ! Lower valueread*,b ! upper valueread*,n ! number of random pointsallocate(R(n))sum=0.0d0do i=1,ncall random_number(R(n))call random_number(R(n+1))X=a+R(n)*(b-a)sum=sum+f(x)int=((b-a)/N)*sumend dowrite(*,*) "integ=",integcontainsreal(8) function F(X)real(8),intent(in)::XF=2*X+1.0d0end function Fend program Sampled_Mean

Generalization to multidimensional cases

The estimate of I is

N

ii

N

ii

XfN

Vxh

NI

11

)()(1ˆ

Rewrite d-dimension integral

dxxfI )( by

dxxxhI )()( Because the uniform distribution ,choose

0)( x

1)( dxxand

Vabx d

iii

1

)(

1)(

1

d

iiiiixVfxfabxh

1

)()()]([)(

Importance Sampled method●It is quite obvious that most of the integral comes from

the region of the peak. But if we generate points evenly in

the interval [a, b], most points won’t be in the peak

area ,and their contribution to the total will be relatively small….

●The idea behind importance sampling is to transformf(x) into another, flatter

function which is then MC

integrated of course there has to be

a back-transformation

to give the original integral

we really want.

Important properties of continuous and discrete pdf ’s

Steps of Importance Sampled method rewrite

b

a

dxxfI )( by b

a

dxxgxhI )()(

where)(

)()(

xg

xfxh

b

a

xdGxhI )()( where x

dxxgxG0

)()(

Now make a variable change)(xGu and )(xdGdu )(1 uGx Then

)(

)(1

1

))((

))((bG

aG

duuGg

uGfI

Cont….

now the value of the integration equal to the average value

N

ii

i

uGg

uGf

NI

11

1

))((

))((1

♠♠♠ note that, the function must be invertible

The error calculated by variance method

)var(Ierror

2

2

2)var(

g

f

g

f

g

f

)(xG

][1

)var( 2

2

2

g

f

g

f

NI

)var(1

)var(g

f

NI

Example (Importance-Sampled method )

find 1

0

2

dxeI x

In this region, the function decreases from 1 to 1/e. The simple exponential function does the same, so let’s

use that for g(x) . We first have to normalize g , so we

xe

calculate

1

0

11

1

e

e

edxe x

Then our normalized weighting function is

1)(

e

eexg

x

Cont....

x x

x

dxe

edxxgxG

0 0

1

1)()(

1

)1()(

e

eexG

x

)(xGu let

)1

1(log)(1e

euuGx

e

Then

)1

1(log)(1e

euuGiei

Cont…

♣♣♣ note:: from the last result we redistribute the PRN according to the pdf (g(x)) , then the new values (i.e., ) are uniform random numbers used

To find the value of

on the interval of integration ,where the average

value is the estimator value for the original integral.

)(1iuG

N

ii

i

uGg

uGf

NI

11

1

))((

))((1

program important_sampleimplicit nonereal(8)::sum,u,R,integinteger::i,Nread*,Nsum=0.0do i=1,Ncall random_number(R)u=G_inv(R)sum=sum+f(u)/g(u)end dointeg=sum/Nwrite(*,*) integcontainsreal(8) function F(x)real(8),intent(in)::xF=dexp(-x**2)end function Freal(8) function g(x)real(8),intent(in)::xg=dexp(-x)end function greal(8) function G_inv(x)real(8),intent(in)::xG_inv=-dlog(1-x*1.718d0/2.718d0)end function G_inv end program important_sample

Why monte carlo ? Method Conv.rate

in one dimConv.rate in d-dim

Basic MCTrapezoidal ruleSimpson’s rule

21

N2N4N

21

NdN2

dN4

MC is from the best methods to find the partition function numerically , then you can solve the stochastic processes.

Suppose we want to solve the following integral Using any other numerical method ,(i.e. Trapezoidal or Simpson)

ii

dxxfI )( Where i=1,2,3,……,30

So we generate a grid in the domain of integration ,supposea grid with 10 nodes on each coordinate axis in 30 dimension cube[ 0,1] ³º be chosen .in this case, we have 10³º abscissas.Suppose a time of 10 ־ second is necessary for calculatingOne value of the function . Therefore , a time of 10¹ years will be necessary for evaluating the integral.

Thank you