técnicas de reducción de varianza

8/18/2019 Técnicas de reducción de varianza

1/49

Reduction variance techniques

Chapter 9 from Ross (2013)

Freddy Hernández BarajasUniversidad Nacional de Colombia

Freddy Hernández BarajasUniversidad Nacional de Colombia Reduction variance techniques

http://find/


2/49

Introduction

Suppose as usual that we wish to estimate θ. Then the standardsimulation algorithm is:

1 Generate U1, U2, . . . , Un.

2

Estimate θ with θ̂n = n

j =1 X j /n where X j = h(U j )3 Approximate 100(1 − α)% confidence intervals are given by

θ̂n − z α/2

σ̂n√ n

, θ̂n + z α/2σ̂n√ n

where σ̂n is the usual estimate of Var (X ) based on X 1,X 2, . . . ,X n.


http://find/


3/49

Introduction

We would like IC to be small, but sometimes this is difficult toachieve. This may be because Var (X ) is too large, or too muchcomputational effort is required to simulate each X j so that n isnecessarily small, or some combination of the two.

There are a number of things we can do:

Develop a good simulation algorithm.

Program carefully to minimize execution time.

Program carefully to minimize storage requirements. Forexample we do not need to store all the X j ’s: we only need to

keep track of

X j and

X 2 j .

Decrease the variability of the simulation output that we use to

estimate θ. The techniques used to do this are usually calledvariance reduction techniques.


http://goforward/http://find/http://goback/


4/49

Variance reduction techniques

1 Antithetic variables.

2 Control variates.3 Conditioning.


http://find/


5/49

1. Antithetic variables

Suppose we are interested in using simulation to estimate θ = E [X ]and suppose we have generated X 1 and X 2, identically distributedrandom variables having mean θ. Then

Var

X 1 + X 2

2

=

1

4 (Var (X 1) + Var (X 2) + 2Cov (X 1,X 2))

Hence it would be adventageous if X 1 and X 2 rather than being

independent were negatively correlated.


http://find/http://goback/


6/49

. . . continuation

To see how we migth arrange for X 1 and X 2 to be negativelycorrelated, suppose that X 1 is a function of m random numbers

X 1 = h(U 1,U 2, . . . ,U m)

where U 1,U 2, . . . ,U m are m independent random numbers.

If U ∼ U (0, 1) then 1 − U is also distribuited as U (0, 1) and

X 2 = h(1 − U 1, 1 − U 2, . . . , 1 − U m)

has the same distribution as X 1 and it is negatively correlated withX 1.

What type of function could be h(·)? Consult page 156.


http://find/


7/49

Example 9d

Suppose we were interested in using simulation to estimate

θ = E [e u ] = 1

0

e x dx

It is apropiate the use of the antithetic variable 1 − u ?We will compare the variance without and with antithetic variablesto decide.


http://find/


8/49

. . . continuation

First

Second


http://find/


9/49

. . . continuation

We see that the use of independent variables results in a variance of

whereas the use of the antithetic variables U and 1 − U gives avariance of


http://find/


10/49

. . . continuation

Without antithetic variables

g


11/49

. . . continuation

Using antithetic variables

x2


12/49

. . . continuation

Comparing the two approaches

0 200 400 600 800 1000

1.5

1.6

1.7

1.8

1.9

2.0

Without antithetic variables

Iterations

M e a n e v o l u t i o n

True integral value

0 200 400 600 800 1000

1.5

1.6

1.7

1.8

1.9

2.0

With antithetic variable

Iterations



ti ti

http://find/


13/49

. . . continuation

f


14/49

. . . continuation

aprox


15/49

. . . continuation

n


16/49

. . . continuation

mean(without.anti)

## [1] 1.718942

sd(without.anti)

## [1] 0.01525596

mean(with.anti)

## [1] 1.718274

sd(with.anti)

## [1] 0.002755573


Example 1

http://find/


17/49

Example 1

Estimate the integral ∞

0 g (x ) dx = ∞

0 log(1 + x 2)e −x dx .

To solve the integral we can use the change variable t = 1 − e −x toensure that f (

·) is a monotonic function.

f (t ) = log(1 + log2(1 − t ))

and the integral to solve is

1

0 f (t ) = log(1 + log2(1 − t ))dt .


continuation

http://find/


18/49

. . . continuation

0 5 10 15

0 .

0 0

0 .

0 5

0 .

1 0

0 .

1 5

0 .

2 0

0 .

2 5

x

g ( x )

0.0 0.2 0.4 0.6 0.8 1.0

0 .

0

0 .

5

1 .

0

1 .

5

2 .

0

2 .

5

3 .

0

x

f ( x )


continuation

http://find/


19/49

. . . continuation

f


20/49

. . . continuation

aprox


21/49

. . . continuation

n


22/49

Example 2

Estimate V = π/4

0

π/40 x

2y 2 sin(x + y ) ln(x + y )dxdy .

If we use x = πu 1/4, y = πu 2/4 the integral limits are 0 and 1.Note that f (u 1, u 2) is monotonic in both variables, for u 1 ∈ [0, 1]and u 2 ∈ [0, 1] .


. . . continuation

http://find/


23/49

. . . continuation

Not using antithetic variables

g


24/49

Using antithetic variables

g


25/49

Comparing results.

0 200 400 600 800 1000

0.000

0.002

0.004

0.006

0.008

0.010

Iterations


Without a.v.

With a.v.


Exercise from Rizzo (2008)

http://find/http://find/


26/49

( )

Use Monte Carlo integration with antithetic variables to estimate

10

e −x

1 + x 2dx ,

and find the approximate reduction in variance as a percentage of the variance without variance reduction.


2. Control variates

http://find/


27/49

Suppose that we want to use simulation to estimate Θ = E [X ],where X is the output of a simulation. Now suppose that for someother output variable Y, the expected value of Y is known-say,E [Y ] = µY .

Then for any constant c , the quantity

X + c (Y − µY )

is also an unbiased estimator of Θ.


. . . continuation



28/49

To determine the best value of c , note that

Simple calculus now shows that the above is minimized whenc = c , where

c = −Cov (X ,Y )Var (Y )


. . . continuation

http://find/


29/49

and for this value the variance of the estimator is

The quantity Y is called a control variate for the simulationestimator X .

The quantity X + c (Y − µY ) is called the controlledestimator.


. . . continuation

http://find/


30/49

Upon dividing the last Equation by Var (X ), we obtain that

is the correlation between X and Y . Hence, the variance reductionobtained in using the control variate Y is 100 Corr 2(X ,Y ) percent.


. . . continuation



31/49

The quantities Cov (X ,Y ) and Var (Y ) are usually not known inadvance and must be estimated from the simulated data. If nsimulation runs are performed, and the output data X i ,Y i , withi = 1, . . . , n, result, then using the estimators


. . . continuation

http://find/


32/49

we can approximate c by ĉ , where


Example 9h



33/49

As in Example 9d, suppose we were interested in using simulation to

compute Θ = E [e U ]. Here, a natural variate to use as a control isthe random number U .


. . . continuation

http://find/


34/49

Because Var (U ) = 1/12 it follows that

where the above used, from Example 9d, that Var (e U ) = 0.2420.Hence, in this case, the use of the control variate U can lead to avariance reduction of up to 98.4 percent.


. . . continuation

http://find/


35/49

We want to estimate E [e U ] until its standard deviation d


36/49

The results are

i

## [1] 2429

mean( x )

## [1] 1.731676


. . . continuation



37/49

Using control variate.

U


38/49

The results are

i

## [1] 43

mean( Z )

## [1] 1.721574


. . . continuation



39/49

Resuming

0 500 1000 1500 2000 2500

1.5

1.6

1.7

1.8

1.9

2.0

Without control variate

Iterations

x M e a n

0 10 20 30 40

1.5

1.6

1.7

1.8

1.9

2.0

With control variate

Iterations

Z M e a n


3. Variance reduction by Conditioning



40/49

Suppose we are interested in performing a simulation study so as to

ascertain the value of Θ = E [X ], where X is an output variable of asimulation run. Also, suppose there is a second variable Y , suchthat E [X

|Y ] is known and takes on a value that can be determined

from the simulation run. Since

E [E [X | Y ]] = E [X ] = Θ

it follows that E [X

|Y ] is also an unbiased estimator of Θ.


. . . continuation

http://find/


41/49

Recall the conditional variance formula proved in Section 2.10 of Chapter 2.

Var (X ) = E [Var (X | Y )] + Var (E [X | Y ])

Since both terms on the right are nonnegative,

Var (X ) ≥ Var (E [X | Y ])

it follows that as an estimator of Θ, E [X | Y ] is superior to the(raw) estimator X .


Example 9k



42/49

Show how we can estimate π by determining how often a randomlychosen point in the square of area 4 centered around the origin falls

within the inscribed circle of radius 1. Specifically, if we letV i = 2U i − 1, where U i , i = 1, 2, are random numbers, and set


. . . continuation



43/49

The estimators I

and (1 −V 2

1 )

12

have mean π/4.


. . . continuation



44/49

Since

it follows that V 21 and U 2 have the same distribution, so we cansimplify by using the estimator (1 − U 2) 12 where U ∼ U (0, 1).What are the variances of I and (1 − U 2) 12 ?


. . . continuation

http://find/


45/49


. . . continuation

http://find/


46/49

We can illustrate the example in R:

n


47/49

0 100 200 300 400 500

0.5

0.6

0.7

0.8

0.9

1.0

1.1

1.2

Mean evolution

Iterations

M e a

n

True value

Without conditioningWith conditioning


. . . continuation

http://find/http://find/http://goback/


48/49

We can obtain the empirical variances of I and (1 − U 2

)

12

.

var(I)

## [1] 0.1638918

var(Z)

## [1] 0.04519196


Homework

http://find/


49/49

Do exercises.

http://find/

técnicas de reducción de varianza

Documents