processi stocastici (3): processi gaussiani -...

of 14/14
Corso di Modelli di Computazione Aettiva Prof. Giuseppe Boccignone Dipartimento di Informatica Università di Milano [email protected] [email protected] http://boccignone.di.unimi.it/CompA2016.html Processi stocastici (3): processi gaussiani Processi stocastici // Classi di processi

Post on 15-Feb-2019

214 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

Corso di Modelli di Computazione Affettiva

Prof. Giuseppe Boccignone

Dipartimento di Informatica

Universit di Milano

[email protected]

[email protected]

http://boccignone.di.unimi.it/CompAff2016.html

Processi stocastici (3): processi gaussiani

Processi stocastici // Classi di processi

Table of content 25

Fig. 14 A conceptual mapof stochastic processes thatare likely to play a role ineye movement modelling andanalyses

.

P(2) = propagator!!!!!!

P(2 | 1) P(1)dx1 (27)

where P(2 | 1) serves as the evolution kernel or propagator P(1 2).A stochastic process whose joint pdf does not change when shifted in time is

called a (strict sense) stationary process:

P(x1, t1; x2, t2; ; xn , tn) = P(x1, t1 + ; x2, t2 + ; ; xn , tn + ) (28)

> 0 being a time shift. Analysis of a stationary process is frequently much simplerthan for a similar process that is time-dependent: varying t, all the random variablesXt have the same law; all the moments, if they exist, are constant in time; the distri-bution of X(t1) and X(t2) depends only on the difference = t2 t1 (time lag), i.e,P(x1, t1; x2, t2) = P(x1,x2; ).

A conceptual map of main kinds of stochastic processes that we will discuss inthe remainder of this Chapter is presented in Figure 14.

5 How to leave the past behind: Markov Processes

The most simple kind of stochastic process is the Purely Random Process in whichthere are no correlations. From Eq. (27):

P(1,2) = P(1)P(2) (29)P(1,2,3) = P(1)P(2)P(3)

P(1,2,3,4) = P(1)P(2)P(3)P1(3)

One such process can be obtained for example by repeated coin tossing. Thecomplete independence property can be written explicitly as:

Processi Gaussiani: prendendo un qualsiasi numero finito di variabili aleatorie, dallensemble che forma il processo aleatorio stesso, esse hanno una distribuzione di probabilit congiunta gaussiana (multivariata)

Pi precisamente: Gaussiano se per qualunque scelta di si ha che

Processi stocastici // Classi di processi

Chapter 3

Gaussian Processes andStochastic DifferentialEquations

3.1 Gaussian Processes

Gaussian processes are a reasonably large class of processes that havemany applications, including in finance, time-series analysis and machinelearning. Certain classes of Gaussian processes can also be thought of asspatial processes. We will use these as a vehicle to start considering moregeneral spatial objects.

Definition 3.1.1 (Gaussian Process). A stochastic process {Xt}t0 is Gaus-sian if, for any choice of times t1, . . . , tn, the random vector (Xt1 , . . . , Xtn)has a multivariate normal distribution.

3.1.1 The Multivariate Normal Distribution

Because Gaussian processes are defined in terms of the multivariate nor-mal distribution, we will need to have a pretty good understanding of thisdistribution and its properties.

Definition 3.1.2 (Multivariate Normal Distribution). A vectorX = (X1, . . . , Xn)is said to be multivariate normal (multivariate Gaussian) if all linear combi-nations of X, i.e. all random variables of the form

n

k=1

kXk

have univariate normal distributions.

89

Chapter 3

Gaussian Processes andStochastic DifferentialEquations

3.1 Gaussian Processes

Gaussian processes are a reasonably large class of processes that havemany applications, including in finance, time-series analysis and machinelearning. Certain classes of Gaussian processes can also be thought of asspatial processes. We will use these as a vehicle to start considering moregeneral spatial objects.

Definition 3.1.1 (Gaussian Process). A stochastic process {Xt}t0 is Gaus-sian if, for any choice of times t1, . . . , tn, the random vector (Xt1 , . . . , Xtn)has a multivariate normal distribution.

3.1.1 The Multivariate Normal Distribution

Because Gaussian processes are defined in terms of the multivariate nor-mal distribution, we will need to have a pretty good understanding of thisdistribution and its properties.

Definition 3.1.2 (Multivariate Normal Distribution). A vectorX = (X1, . . . , Xn)is said to be multivariate normal (multivariate Gaussian) if all linear combi-nations of X, i.e. all random variables of the form

n

k=1

kXk

have univariate normal distributions.

89

3.1. GAUSSIAN PROCESSES 93

Simulating Multivariate Normals

One possible way to simulate a multivariate normal random vector isusing the Cholesky decomposition.

Lemma 3.1.11. If Z N(0, I), = AA is a covariance matrix, andX = + AZ, then X N(,).

Proof. We know from lemma 3.1.4 that AZ is multivariate normal and sois + AZ. Because a multivariate normal random vector is completelydescribed by its mean vector and covariance matrix, we simple need to findEX and Var(X). Now,

EX = E+ EAZ = + A0 =

and

Var(X) = Var(+ AZ) = Var(AZ) = AVar(Z)A = AIA = AA = .

Thus, we can simulate a random vector X N(,) by

(i) Finding A such that = AA.

(ii) Generating Z N(0, I).

(iii) Returning X = + AZ.

Matlab makes things a bit confusing because its function chol producesa decomposition of the form BB. That is, A = B. It is important tobe careful and think about whether you want to generate a row or columnvector when generating multivariate normals.

The following code produces column vectors.

Listing 3.1: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + A * randn(3,1);

The following code produces row vectors.

Listing 3.2: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + randn(1,3) * A;

92 CHAPTER 3. GAUSSIAN PROCESSES ETC.

If A is also symmetric, then A is called symmetric positive semi-definite(SPSD). Note that if A is an SPSD then there is a real-valued decomposition

A = LL,

though it is not necessarily unique and L may have zeroes on the diagonals.

Lemma 3.1.9. Covariance matrices are SPSD.

Proof. Given an n 1 real-valued vector x and a random vector Y withcovariance matrix , we have

Var (xY) = xVar (Y)x.

Now this must be non-negative (as it is a variance). That is, it must be thecase that

xVar (Y)x 0,

so is positive semi-definite. Symmetry comes from the fact that Cov(X, Y ) =Cov(Y,X).

Lemma 3.1.10. SPSD matrices are covariance matrices.

Proof. Let A be an SPSD matrix and Z be a vector of random variables withVar(Z) = I. Now, as A is SPSD, A = LL. So,

Var(LZ) = LVar(Z)L = LIL = A,

so A is the covariance matrix of LZ.

COVARIANCE POS DEF ...

Densities of Multivariate Normals

If X = (X1, . . . , Xn) is N(,) and is positive definite, then X has thedensity

f(x) = f(x1, . . . , xn) =1

(2)n||exp

{12(x )1(x )

}.

Chapter 3

Gaussian Processes andStochastic DifferentialEquations

3.1 Gaussian Processes

Gaussian processes are a reasonably large class of processes that havemany applications, including in finance, time-series analysis and machinelearning. Certain classes of Gaussian processes can also be thought of asspatial processes. We will use these as a vehicle to start considering moregeneral spatial objects.

Definition 3.1.1 (Gaussian Process). A stochastic process {Xt}t0 is Gaus-sian if, for any choice of times t1, . . . , tn, the random vector (Xt1 , . . . , Xtn)has a multivariate normal distribution.

3.1.1 The Multivariate Normal Distribution

Because Gaussian processes are defined in terms of the multivariate nor-mal distribution, we will need to have a pretty good understanding of thisdistribution and its properties.

Definition 3.1.2 (Multivariate Normal Distribution). A vectorX = (X1, . . . , Xn)is said to be multivariate normal (multivariate Gaussian) if all linear combi-nations of X, i.e. all random variables of the form

n

k=1

kXk

have univariate normal distributions.

89

92 CHAPTER 3. GAUSSIAN PROCESSES ETC.

If A is also symmetric, then A is called symmetric positive semi-definite(SPSD). Note that if A is an SPSD then there is a real-valued decomposition

A = LL,

though it is not necessarily unique and L may have zeroes on the diagonals.

Lemma 3.1.9. Covariance matrices are SPSD.

Proof. Given an n 1 real-valued vector x and a random vector Y withcovariance matrix , we have

Var (xY) = xVar (Y)x.

Now this must be non-negative (as it is a variance). That is, it must be thecase that

xVar (Y)x 0,

so is positive semi-definite. Symmetry comes from the fact that Cov(X, Y ) =Cov(Y,X).

Lemma 3.1.10. SPSD matrices are covariance matrices.

Proof. Let A be an SPSD matrix and Z be a vector of random variables withVar(Z) = I. Now, as A is SPSD, A = LL. So,

Var(LZ) = LVar(Z)L = LIL = A,

so A is the covariance matrix of LZ.

COVARIANCE POS DEF ...

Densities of Multivariate Normals

If X = (X1, . . . , Xn) is N(,) and is positive definite, then X has thedensity

f(x) = f(x1, . . . , xn) =1

(2)n||exp

{12(x )1(x )

}.

92 CHAPTER 3. GAUSSIAN PROCESSES ETC.

If A is also symmetric, then A is called symmetric positive semi-definite(SPSD). Note that if A is an SPSD then there is a real-valued decomposition

A = LL,

though it is not necessarily unique and L may have zeroes on the diagonals.

Lemma 3.1.9. Covariance matrices are SPSD.

Proof. Given an n 1 real-valued vector x and a random vector Y withcovariance matrix , we have

Var (xY) = xVar (Y)x.

Now this must be non-negative (as it is a variance). That is, it must be thecase that

xVar (Y)x 0,

so is positive semi-definite. Symmetry comes from the fact that Cov(X, Y ) =Cov(Y,X).

Lemma 3.1.10. SPSD matrices are covariance matrices.

Proof. Let A be an SPSD matrix and Z be a vector of random variables withVar(Z) = I. Now, as A is SPSD, A = LL. So,

Var(LZ) = LVar(Z)L = LIL = A,

so A is the covariance matrix of LZ.

COVARIANCE POS DEF ...

Densities of Multivariate Normals

If X = (X1, . . . , Xn) is N(,) and is positive definite, then X has thedensity

f(x) = f(x1, . . . , xn) =1

(2)n||exp

{12(x )1(x )

}.

92 CHAPTER 3. GAUSSIAN PROCESSES ETC.

If A is also symmetric, then A is called symmetric positive semi-definite(SPSD). Note that if A is an SPSD then there is a real-valued decomposition

A = LL,

though it is not necessarily unique and L may have zeroes on the diagonals.

Lemma 3.1.9. Covariance matrices are SPSD.

Proof. Given an n 1 real-valued vector x and a random vector Y withcovariance matrix , we have

Var (xY) = xVar (Y)x.

Now this must be non-negative (as it is a variance). That is, it must be thecase that

xVar (Y)x 0,

so is positive semi-definite. Symmetry comes from the fact that Cov(X, Y ) =Cov(Y,X).

Lemma 3.1.10. SPSD matrices are covariance matrices.

Proof. Let A be an SPSD matrix and Z be a vector of random variables withVar(Z) = I. Now, as A is SPSD, A = LL. So,

Var(LZ) = LVar(Z)L = LIL = A,

so A is the covariance matrix of LZ.

COVARIANCE POS DEF ...

Densities of Multivariate Normals

If X = (X1, . . . , Xn) is N(,) and is positive definite, then X has thedensity

f(x) = f(x1, . . . , xn) =1

(2)n||exp

{12(x )1(x )

}.

Processi stocastici // Classi di processi

Processi Gaussiani: prendendo un qualsiasi numero finito di variabili aleatorie, dalla collezione che forma il processo aleatorio stesso, esse hanno una distribuzione di probabilit congiunta gaussiana (multivariata)

Sono processi continui nel tempo e nello spazio degli stati

Esempio: il rumore bianco un processo Gaussiano i.i.d

90 CHAPTER 3. GAUSSIAN PROCESSES ETC.

This is quite a strong definition. Importantly, it implies that, even if all ofits components are normally distributed, a random vector is not necessarilymultivariate normal.

Example 3.1.3 (A random vector with normal marginals that is not multi-variate normal). Let X1 N(0, 1) and

X2 =

{X1 if |X1| 1X1 if |X1| > 1

.

Note that X2 N(0, 1). However, X1 + X2 is not normally distributed,because |X1+X2| 2, which implies X1+X2 is bounded and, hence, cannotbe normally distributed.

Linear transformations of multivariate normal random vectors are, again,multivariate normal.

Lemma 3.1.4. Suppose X = (X1, . . . , Xn) is multivariate normal and A isan m n real-valued matrix. Then, Y = AX is also multivariate normal.Proof. Any linear combination of Y1, . . . , Ym is a linear combination of linearcombinations of X1, . . . , Xn and, thus, univariate normal.

Theorem 3.1.5. A multivariate normal random vector X = (X1, . . . , Xn)is completely described by a mean vector = EX and a covariance matrix = Var(X).

Proof. The distribution of X is described by its characteristic function whichis

Eexp {iX} = Eexp{

in

i=1

iXi

}

.

Now, we known

i=1 iXi is a univariate normal random variable (becauseX is multivariate normal). Let m = E

ni=1 iXi and

2 = Var (n

i=1 iXi).Then,

E

{

in

i=1

iXi

}

= exp

{im 1

22}.

Now

m = En

i=1

iXi =n

i=1

ii

and

2 = Var

(n

i=1

iXi

)

=n

i=1

n

j=1

ijCov(Xi, Xj) =n

j=1

n

j=1

iji,j .

So, everything is specified by and .

Processi stocastici // Classi di processi

Processi Gaussiani: prendendo un qualsiasi numero finito di variabili aleatorie, dalla collezione che forma il processo aleatorio stesso, esse hanno una distribuzione di probabilit congiunta gaussiana (multivariata)

Un processo Gaussiano completamente specificato mediante media e matrice di covarianza

3.1. GAUSSIAN PROCESSES 93

Simulating Multivariate Normals

One possible way to simulate a multivariate normal random vector isusing the Cholesky decomposition.

Lemma 3.1.11. If Z N(0, I), = AA is a covariance matrix, andX = + AZ, then X N(,).

Proof. We know from lemma 3.1.4 that AZ is multivariate normal and sois + AZ. Because a multivariate normal random vector is completelydescribed by its mean vector and covariance matrix, we simple need to findEX and Var(X). Now,

EX = E+ EAZ = + A0 =

and

Var(X) = Var(+ AZ) = Var(AZ) = AVar(Z)A = AIA = AA = .

Thus, we can simulate a random vector X N(,) by

(i) Finding A such that = AA.

(ii) Generating Z N(0, I).

(iii) Returning X = + AZ.

Matlab makes things a bit confusing because its function chol producesa decomposition of the form BB. That is, A = B. It is important tobe careful and think about whether you want to generate a row or columnvector when generating multivariate normals.

The following code produces column vectors.

Listing 3.1: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + A * randn(3,1);

The following code produces row vectors.

Listing 3.2: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + randn(1,3) * A;

Processi stocastici // Classi di processi

Processi Gaussiani: moto Browniano

Pu essere considerato il limite continuo del random walk semplice

Processi stocastici // Classi di processi

Table of content 25

Fig. 14 A conceptual mapof stochastic processes thatare likely to play a role ineye movement modelling andanalyses

.

P(2) = propagator!!!!!!

P(2 | 1) P(1)dx1 (27)

where P(2 | 1) serves as the evolution kernel or propagator P(1 2).A stochastic process whose joint pdf does not change when shifted in time is

called a (strict sense) stationary process:

P(x1, t1; x2, t2; ; xn , tn) = P(x1, t1 + ; x2, t2 + ; ; xn , tn + ) (28)

> 0 being a time shift. Analysis of a stationary process is frequently much simplerthan for a similar process that is time-dependent: varying t, all the random variablesXt have the same law; all the moments, if they exist, are constant in time; the distri-bution of X(t1) and X(t2) depends only on the difference = t2 t1 (time lag), i.e,P(x1, t1; x2, t2) = P(x1,x2; ).

A conceptual map of main kinds of stochastic processes that we will discuss inthe remainder of this Chapter is presented in Figure 14.

5 How to leave the past behind: Markov Processes

The most simple kind of stochastic process is the Purely Random Process in whichthere are no correlations. From Eq. (27):

P(1,2) = P(1)P(2) (29)P(1,2,3) = P(1)P(2)P(3)

P(1,2,3,4) = P(1)P(2)P(3)P1(3)

One such process can be obtained for example by repeated coin tossing. Thecomplete independence property can be written explicitly as:

Processi stocastici // Classi di processi

Processi Gaussiani: moto Browniano

Pu essere considerato il limite continuo del random walk semplice

2.2. IL MOTO BROWNIANO 27

la formalizzazione precisa di questo risultato alla Proposizione 2.25, dopo che avremodiscusso la nozione di -algebra associata a un processo.

2.2. Il moto browniano

Ricordiamo lOsservazione 1.2: fissato uno spazio di probabilit (,F ,P), scriveremo q.c.[. . . ] come abbreviazione di esiste A 2 F , con P(A) = 1, tale che per ogni ! 2 A [. . . ].

Definiamo ora il moto browniano, detto anche processo di Wiener, che costituisceloggetto centrale di questo corso. Si tratta dellesempio pi importante di processostocastico a tempo continuo. Esso pu essere visto come lanalogo a tempo continuo diuna passeggiata aleatoria reale con incrementi gaussiani. In effetti, come discuteremo piavanti, il moto browniano pu essere ottenuto come un opportuno limite di qualunquepasseggiata aleatoria con incrementi di varianza finita (cf. il sottoparagrafo 2.7.1).

Definizione 2.3 (Moto browniano). Si dice moto browniano qualunque processostocastico reale B = {B

t

}t2[0,1) che soddisfa le seguenti propriet:

(a) B0

= 0 q.c.;(b) B ha incrementi indipendenti, cio per ogni scelta di k 2 e 0 t

0

< t1

< . . . s 0 si ha (Bt

Bs

) N (0, t s);(d) q.c. B ha traiettorie continue, cio q.c. la funzione t 7! B

t

continua.

Nella definizione sottinteso lo spazio di probabilit (,F ,P) su cui definito ilprocesso B, per cui si ha B

t

= Bt

(!) con ! 2 . La dipendenza da ! verr quasi sempreomessa, ma importante essere in grado di esplicitarla quando necessario. Per esempio,la propriet (d) si pu riformulare nel modo seguente: esiste A 2 F con P(A) = 1 taleche per ogni ! 2 A la funzione t 7! B

t

(!) continua. Oltre a essere una richiesta moltonaturale dal punto di vista fisico, la continuit delle traiettorie una propriet di basilareimportanza anche da un punto di vista matematico (si veda il sottoparagrafo 2.2.2).

Talvolta parleremo di moto browniano con insieme dei tempi ristretto a un intervalloT = [0, t

0

], dove t0

2 (0,1) fissato, intendendo naturalmente con ci un processo{B

t

}t2T che soddisfa le condizioni della Definizione 2.3 per t ristretto a T.

Nella Figura 2.1 sono mostrate tre traiettorie illustrative del moto browniano.Veniamo ora al primo risultato fondamentale sul moto browniano, dimostrato per la

prima volta da Norbert Wiener nel 1923. A dispetto delle apparenze, si tratta di unrisultato non banale.

Teorema 2.4 (Wiener). Il moto browniano esiste.

Sono possibili diverse dimostrazioni di questo teorema. Un metodo standard, basatosu un teorema molto generale dovuto a Kolmogorov, consiste nel costruire sullo spazio

2.2. IL MOTO BROWNIANO 27

la formalizzazione precisa di questo risultato alla Proposizione 2.25, dopo che avremodiscusso la nozione di -algebra associata a un processo.

2.2. Il moto browniano

Ricordiamo lOsservazione 1.2: fissato uno spazio di probabilit (,F ,P), scriveremo q.c.[. . . ] come abbreviazione di esiste A 2 F , con P(A) = 1, tale che per ogni ! 2 A [. . . ].

Definiamo ora il moto browniano, detto anche processo di Wiener, che costituisceloggetto centrale di questo corso. Si tratta dellesempio pi importante di processostocastico a tempo continuo. Esso pu essere visto come lanalogo a tempo continuo diuna passeggiata aleatoria reale con incrementi gaussiani. In effetti, come discuteremo piavanti, il moto browniano pu essere ottenuto come un opportuno limite di qualunquepasseggiata aleatoria con incrementi di varianza finita (cf. il sottoparagrafo 2.7.1).

Definizione 2.3 (Moto browniano). Si dice moto browniano qualunque processostocastico reale B = {B

t

}t2[0,1) che soddisfa le seguenti propriet:

(a) B0

= 0 q.c.;(b) B ha incrementi indipendenti, cio per ogni scelta di k 2 e 0 t

0

< t1

< . . . s 0 si ha (Bt

Bs

) N (0, t s);(d) q.c. B ha traiettorie continue, cio q.c. la funzione t 7! B

t

continua.

Nella definizione sottinteso lo spazio di probabilit (,F ,P) su cui definito ilprocesso B, per cui si ha B

t

= Bt

(!) con ! 2 . La dipendenza da ! verr quasi sempreomessa, ma importante essere in grado di esplicitarla quando necessario. Per esempio,la propriet (d) si pu riformulare nel modo seguente: esiste A 2 F con P(A) = 1 taleche per ogni ! 2 A la funzione t 7! B

t

(!) continua. Oltre a essere una richiesta moltonaturale dal punto di vista fisico, la continuit delle traiettorie una propriet di basilareimportanza anche da un punto di vista matematico (si veda il sottoparagrafo 2.2.2).

Talvolta parleremo di moto browniano con insieme dei tempi ristretto a un intervalloT = [0, t

0

], dove t0

2 (0,1) fissato, intendendo naturalmente con ci un processo{B

t

}t2T che soddisfa le condizioni della Definizione 2.3 per t ristretto a T.

Nella Figura 2.1 sono mostrate tre traiettorie illustrative del moto browniano.Veniamo ora al primo risultato fondamentale sul moto browniano, dimostrato per la

prima volta da Norbert Wiener nel 1923. A dispetto delle apparenze, si tratta di unrisultato non banale.

Teorema 2.4 (Wiener). Il moto browniano esiste.

Sono possibili diverse dimostrazioni di questo teorema. Un metodo standard, basatosu un teorema molto generale dovuto a Kolmogorov, consiste nel costruire sullo spazio

26 2. MOTO BROWNIANO

prodotto (EI , EI), che detto talvolta spazio delle traiettorie del processo X. Comeogni variabile aleatoria, X induce sullo spazio darrivo (EI , EI) la sua legge

X

: questaprobabilit detta legge del processo.

Se C = {x 2 EI : xt1 2 A1, . . . , xt

k

2 Ak

} un sottoinsieme cilindrico di EI , siha

X

(C) = P(X 2 C) = P((Xt1 , . . . , Xt

k

) 2 A1

Ak

), dunque la probabilitX

(C) pu essere calcolata conoscendo le leggi finito-dimensionali di X. Ricordando che isottoinsiemi cilindrici sono una base della -algebra EI , segue che la legge

X

del processoX sullo spazio delle traiettorie (EI , EI) determinata dalle leggi finito-dimensionali di X.(Per questa ragione, con il termine legge del processo X si indica talvolta la famiglia delleleggi finito dimensionali.) In particolare, due processi X = {X

t

}t2I , X 0 = {X 0

t

}t2I con lo

stesso insieme degli indici I e a valori nello stesso spazio (E, E) hanno la stessa legge se esolo se hanno le stesse leggi finito-dimensionali.

2.1.3. Processi gaussiani. Un processo vettoriale X = {Xt

}t2I a valori in Rd, con

Xt

= (X(1)t

, . . . , X(d)t

), pu essere sempre visto come un processo stocastico reale a pattodi ampliare linsieme degli indici: infatti basta scrivere X = {X(i)

t

}(i,t)2{1,...,d}I . Per

questa ragione, quando risulta conveniente, possibile limitare la trattazione ai processireali, senza perdita di generalit. Questo quello che faremo sempre nel caso dei processigaussiani, che ora definiamo.

Definizione 2.2. Un processo stocastico reale X = {Xt

}t2I detto gaussiano se,

per ogni scelta di t1

, . . . , tn

2 I, il vettore aleatorio (Xt1 , . . . , Xtn) normale, cio se

qualunque combinazione lineare finita delle Xt

una variabile aleatoria normale.

I processi gaussiani costituiscono una generalizzazione dei vettori aleatori normali.Si noti infatti che, quando I = {t

1

, . . . , tk

} un insieme finito, un processo gaussianoX = {X

t

}t2I = (Xt1 , . . . , Xt

k

) non altro che un vettore aleatorio normale a valori in Rk.Come per i vettori normali, dato un processo gaussiano X = {X

t

}t2I introduciamo le

funzioni media (t) := E(Xt

) e covarianza K(s, t) := Cov(Xs

, Xt

), ben definite in quantoX

t

2 L2 per ogni t 2 I (perch?). Si noti che la funzione K(, ) simmetrica e semi-definita positiva, nel senso seguente: per ogni scelta di n 2 N, t

1

, . . . , tn

2 I e di u 2 Rn siha

P

n

i,j=1

K(ti

, tj

)ui

uj

0; infatti {Kij

:= K(ti

, tj

)}1i,jn la matrice di covarianza

del vettore (Xt1 , . . . , Xtn). Si pu mostrare (non lo faremo) che, assegnate arbitrariamente

due funzioni : I ! R e K : I I ! R, con K simmetrica e semi-definita positiva, esisteun processo gaussiano {X

t

}t2I che ha e K come funzioni media e covarianza.

Una propriet fondamentale che le leggi finito-dimensionali di un processo gaussianosono univocamente determinate dalle sue funzioni media () e covarianza K(, ). Questosegue immediatamente dal fatto che ogni vettore della forma (X

t1 , . . . , Xtk

) per defini-zione normale a valori in Rk e dunque la sua funzione caratteristica, espressa dalla formula(1.9), una funzione del vettore ((t

1

), . . . , (tk

)) e della matrice {Kij

:= K(ti

, tj

)}1i,jk.

Anche la propriet basilare per cui variabili congiuntamente normali sono indipendentise e solo se sono scorrelate, cf. il Lemma 1.14, si estende ai processi gaussiani. Rimandiamo

2.2. IL MOTO BROWNIANO 27

la formalizzazione precisa di questo risultato alla Proposizione 2.25, dopo che avremodiscusso la nozione di -algebra associata a un processo.

2.2. Il moto browniano

Ricordiamo lOsservazione 1.2: fissato uno spazio di probabilit (,F ,P), scriveremo q.c.[. . . ] come abbreviazione di esiste A 2 F , con P(A) = 1, tale che per ogni ! 2 A [. . . ].

Definiamo ora il moto browniano, detto anche processo di Wiener, che costituisceloggetto centrale di questo corso. Si tratta dellesempio pi importante di processostocastico a tempo continuo. Esso pu essere visto come lanalogo a tempo continuo diuna passeggiata aleatoria reale con incrementi gaussiani. In effetti, come discuteremo piavanti, il moto browniano pu essere ottenuto come un opportuno limite di qualunquepasseggiata aleatoria con incrementi di varianza finita (cf. il sottoparagrafo 2.7.1).

Definizione 2.3 (Moto browniano). Si dice moto browniano qualunque processostocastico reale B = {B

t

}t2[0,1) che soddisfa le seguenti propriet:

(a) B0

= 0 q.c.;(b) B ha incrementi indipendenti, cio per ogni scelta di k 2 e 0 t

0

< t1

< . . . s 0 si ha (Bt

Bs

) N (0, t s);(d) q.c. B ha traiettorie continue, cio q.c. la funzione t 7! B

t

continua.

Nella definizione sottinteso lo spazio di probabilit (,F ,P) su cui definito ilprocesso B, per cui si ha B

t

= Bt

(!) con ! 2 . La dipendenza da ! verr quasi sempreomessa, ma importante essere in grado di esplicitarla quando necessario. Per esempio,la propriet (d) si pu riformulare nel modo seguente: esiste A 2 F con P(A) = 1 taleche per ogni ! 2 A la funzione t 7! B

t

(!) continua. Oltre a essere una richiesta moltonaturale dal punto di vista fisico, la continuit delle traiettorie una propriet di basilareimportanza anche da un punto di vista matematico (si veda il sottoparagrafo 2.2.2).

Talvolta parleremo di moto browniano con insieme dei tempi ristretto a un intervalloT = [0, t

0

], dove t0

2 (0,1) fissato, intendendo naturalmente con ci un processo{B

t

}t2T che soddisfa le condizioni della Definizione 2.3 per t ristretto a T.

Nella Figura 2.1 sono mostrate tre traiettorie illustrative del moto browniano.Veniamo ora al primo risultato fondamentale sul moto browniano, dimostrato per la

prima volta da Norbert Wiener nel 1923. A dispetto delle apparenze, si tratta di unrisultato non banale.

Teorema 2.4 (Wiener). Il moto browniano esiste.

Sono possibili diverse dimostrazioni di questo teorema. Un metodo standard, basatosu un teorema molto generale dovuto a Kolmogorov, consiste nel costruire sullo spazio

26 2. MOTO BROWNIANO

prodotto (EI , EI), che detto talvolta spazio delle traiettorie del processo X. Comeogni variabile aleatoria, X induce sullo spazio darrivo (EI , EI) la sua legge

X

: questaprobabilit detta legge del processo.

Se C = {x 2 EI : xt1 2 A1, . . . , xt

k

2 Ak

} un sottoinsieme cilindrico di EI , siha

X

(C) = P(X 2 C) = P((Xt1 , . . . , Xt

k

) 2 A1

Ak

), dunque la probabilitX

(C) pu essere calcolata conoscendo le leggi finito-dimensionali di X. Ricordando che isottoinsiemi cilindrici sono una base della -algebra EI , segue che la legge

X

del processoX sullo spazio delle traiettorie (EI , EI) determinata dalle leggi finito-dimensionali di X.(Per questa ragione, con il termine legge del processo X si indica talvolta la famiglia delleleggi finito dimensionali.) In particolare, due processi X = {X

t

}t2I , X 0 = {X 0

t

}t2I con lo

stesso insieme degli indici I e a valori nello stesso spazio (E, E) hanno la stessa legge se esolo se hanno le stesse leggi finito-dimensionali.

2.1.3. Processi gaussiani. Un processo vettoriale X = {Xt

}t2I a valori in Rd, con

Xt

= (X(1)t

, . . . , X(d)t

), pu essere sempre visto come un processo stocastico reale a pattodi ampliare linsieme degli indici: infatti basta scrivere X = {X(i)

t

}(i,t)2{1,...,d}I . Per

questa ragione, quando risulta conveniente, possibile limitare la trattazione ai processireali, senza perdita di generalit. Questo quello che faremo sempre nel caso dei processigaussiani, che ora definiamo.

Definizione 2.2. Un processo stocastico reale X = {Xt

}t2I detto gaussiano se,

per ogni scelta di t1

, . . . , tn

2 I, il vettore aleatorio (Xt1 , . . . , Xtn) normale, cio se

qualunque combinazione lineare finita delle Xt

una variabile aleatoria normale.

I processi gaussiani costituiscono una generalizzazione dei vettori aleatori normali.Si noti infatti che, quando I = {t

1

, . . . , tk

} un insieme finito, un processo gaussianoX = {X

t

}t2I = (Xt1 , . . . , Xt

k

) non altro che un vettore aleatorio normale a valori in Rk.Come per i vettori normali, dato un processo gaussiano X = {X

t

}t2I introduciamo le

funzioni media (t) := E(Xt

) e covarianza K(s, t) := Cov(Xs

, Xt

), ben definite in quantoX

t

2 L2 per ogni t 2 I (perch?). Si noti che la funzione K(, ) simmetrica e semi-definita positiva, nel senso seguente: per ogni scelta di n 2 N, t

1

, . . . , tn

2 I e di u 2 Rn siha

P

n

i,j=1

K(ti

, tj

)ui

uj

0; infatti {Kij

:= K(ti

, tj

)}1i,jn la matrice di covarianza

del vettore (Xt1 , . . . , Xtn). Si pu mostrare (non lo faremo) che, assegnate arbitrariamente

due funzioni : I ! R e K : I I ! R, con K simmetrica e semi-definita positiva, esisteun processo gaussiano {X

t

}t2I che ha e K come funzioni media e covarianza.

Una propriet fondamentale che le leggi finito-dimensionali di un processo gaussianosono univocamente determinate dalle sue funzioni media () e covarianza K(, ). Questosegue immediatamente dal fatto che ogni vettore della forma (X

t1 , . . . , Xtk

) per defini-zione normale a valori in Rk e dunque la sua funzione caratteristica, espressa dalla formula(1.9), una funzione del vettore ((t

1

), . . . , (tk

)) e della matrice {Kij

:= K(ti

, tj

)}1i,jk.

Anche la propriet basilare per cui variabili congiuntamente normali sono indipendentise e solo se sono scorrelate, cf. il Lemma 1.14, si estende ai processi gaussiani. Rimandiamo

Moto Browniano: gli incrementi sono genericamente Gaussiani

Processo di Wiener: se gli incrementi sono Gaussiani centrati (media 0, varianza 1)

108 CHAPTER 3. GAUSSIAN PROCESSES ETC.

Example 3.1.28 (Ornstein-Uhlenbeck Process). The Ornstein-Uhlenbeckprocess has

i = EXti = 0

withi,i+1 = exp{|ti ti+1|/2}

and i,i = 1. So, our updating formula is

Xti+1 = exp{|ti ti+1|/2}Xti +(

1 exp{|ti ti+1|})Z.

Listing 3.9: Matlab code

1 alpha = 50; m = 10^4;

2

3 X = zeros(m,1); h = 1/m;

4 X(1) = randn;

5

6 for i = 1:m-1

7 X(i+1) = exp(-alpha * h / 2)*X(i)...

8 + sqrt(1 - exp(-alpha * h))*randn;

9 end

10

11 plot(h:h:1,X);

3.2 Brownian Motion

One of the most fundamental stochastic processes. It is a Gaussian pro-cess, a Markov process, a Levy process, a Martingale and a process that isclosely linked to the study of harmonic functions (do not worry if you do notknow all these terms). It can be used as a building block when consideringmany more complicated processes. For this reason, we will consider it inmuch more depth than any other Gaussian process.

Definition 3.2.1 (Brownian Motion). A stochastic process {Wt}t0 is calledBrownian motion (a Wiener process) if:

(i) It has independent increments.

(ii) It has stationary increments.

(iii) Wt N(0, t) for all t 0.

Poich gli incrementi sono Gaussiani

Processi stocastici // Classi di processi

Processi Gaussiani: moto Browniano

Pu essere considerato il limite continuo del random walk semplice

116 Stochastic Simulation and Applications in Finance

7.3.3 Wiener Process

Let us examine the limit case of a random walk

X (t) =g(t)

k=1Bk, where g(t) = Int

(t

!t

). (7.44)

We have seen that mX(t) = 0 and 2X (t) = g(t)2. Suppose that the jump is proportional to theduration of the interval !t, =

!t , then 2X (t) = g(t)!t. When !t goes to 0, g(t)!t tends

to t. Thus, taking the limit,

2X (t) = t. (7.45)

On the other hand, when !t goes to 0, X(t) is the sum of a large number of independentbinary random variables. By virtue of the central limit theorem, it converges to a Gaussianrandom variable.

Taken to the limit, the random walk tends toward a Gaussian random process with zeromean, variance 2X (t) = t and independent increments. Such a process is by definition a standardWiener process.

Definition 3.1 W(t) is a standard Wiener process if

(i) W(0) = 0,(ii) W(t) has independent increments, which means that

t1 < t2 < t3 < t4, W (t4) W (t3) and W (t2) W (t1)

are independent,(iii) Z(t) = W(t) W(t0) is Gaussian with zero mean and variance

2Z = t t0, t0 t.

From this definition we can deduce the following basic properties for a standard Wienerprocess.

Property 3.2 Var(W(t)) = t and mW(t) = E[W(t)] = 0. Since its variance increases with timet, the trajectory of this process moves away from the horizontal axis when t increases.

Property 3.3 From the law of large numbers, we can prove that:

W (t)t

t

0. (7.46)

108 CHAPTER 3. GAUSSIAN PROCESSES ETC.

Example 3.1.28 (Ornstein-Uhlenbeck Process). The Ornstein-Uhlenbeckprocess has

i = EXti = 0

withi,i+1 = exp{|ti ti+1|/2}

and i,i = 1. So, our updating formula is

Xti+1 = exp{|ti ti+1|/2}Xti +(

1 exp{|ti ti+1|})Z.

Listing 3.9: Matlab code

1 alpha = 50; m = 10^4;

2

3 X = zeros(m,1); h = 1/m;

4 X(1) = randn;

5

6 for i = 1:m-1

7 X(i+1) = exp(-alpha * h / 2)*X(i)...

8 + sqrt(1 - exp(-alpha * h))*randn;

9 end

10

11 plot(h:h:1,X);

3.2 Brownian Motion

One of the most fundamental stochastic processes. It is a Gaussian pro-cess, a Markov process, a Levy process, a Martingale and a process that isclosely linked to the study of harmonic functions (do not worry if you do notknow all these terms). It can be used as a building block when consideringmany more complicated processes. For this reason, we will consider it inmuch more depth than any other Gaussian process.

Definition 3.2.1 (Brownian Motion). A stochastic process {Wt}t0 is calledBrownian motion (a Wiener process) if:

(i) It has independent increments.

(ii) It has stationary increments.

(iii) Wt N(0, t) for all t 0.

116 Stochastic Simulation and Applications in Finance

7.3.3 Wiener Process

Let us examine the limit case of a random walk

X (t) =g(t)

k=1Bk, where g(t) = Int

(t

!t

). (7.44)

We have seen that mX(t) = 0 and 2X (t) = g(t)2. Suppose that the jump is proportional to theduration of the interval !t, =

!t , then 2X (t) = g(t)!t. When !t goes to 0, g(t)!t tends

to t. Thus, taking the limit,

2X (t) = t. (7.45)

On the other hand, when !t goes to 0, X(t) is the sum of a large number of independentbinary random variables. By virtue of the central limit theorem, it converges to a Gaussianrandom variable.

Taken to the limit, the random walk tends toward a Gaussian random process with zeromean, variance 2X (t) = t and independent increments. Such a process is by definition a standardWiener process.

Definition 3.1 W(t) is a standard Wiener process if

(i) W(0) = 0,(ii) W(t) has independent increments, which means that

t1 < t2 < t3 < t4, W (t4) W (t3) and W (t2) W (t1)

are independent,(iii) Z(t) = W(t) W(t0) is Gaussian with zero mean and variance

2Z = t t0, t0 t.

From this definition we can deduce the following basic properties for a standard Wienerprocess.

Property 3.2 Var(W(t)) = t and mW(t) = E[W(t)] = 0. Since its variance increases with timet, the trajectory of this process moves away from the horizontal axis when t increases.

Property 3.3 From the law of large numbers, we can prove that:

W (t)t

t

0. (7.46)

116 Stochastic Simulation and Applications in Finance

7.3.3 Wiener Process

Let us examine the limit case of a random walk

X (t) =g(t)

k=1Bk, where g(t) = Int

(t

!t

). (7.44)

We have seen that mX(t) = 0 and 2X (t) = g(t)2. Suppose that the jump is proportional to theduration of the interval !t, =

!t , then 2X (t) = g(t)!t. When !t goes to 0, g(t)!t tends

to t. Thus, taking the limit,

2X (t) = t. (7.45)

On the other hand, when !t goes to 0, X(t) is the sum of a large number of independentbinary random variables. By virtue of the central limit theorem, it converges to a Gaussianrandom variable.

Taken to the limit, the random walk tends toward a Gaussian random process with zeromean, variance 2X (t) = t and independent increments. Such a process is by definition a standardWiener process.

Definition 3.1 W(t) is a standard Wiener process if

(i) W(0) = 0,(ii) W(t) has independent increments, which means that

t1 < t2 < t3 < t4, W (t4) W (t3) and W (t2) W (t1)

are independent,(iii) Z(t) = W(t) W(t0) is Gaussian with zero mean and variance

2Z = t t0, t0 t.

From this definition we can deduce the following basic properties for a standard Wienerprocess.

Property 3.2 Var(W(t)) = t and mW(t) = E[W(t)] = 0. Since its variance increases with timet, the trajectory of this process moves away from the horizontal axis when t increases.

Property 3.3 From the law of large numbers, we can prove that:

W (t)t

t

0. (7.46)

La varianza cresce linearmente nel tempo e il processo si sposta lontano dallasse t

La funzione di auto-correlazione

Processi stocastici // Classi di processi

Processi Gaussiani: moto Browniano

Pu essere considerato il limite continuo del random walk semplice

108 CHAPTER 3. GAUSSIAN PROCESSES ETC.

Example 3.1.28 (Ornstein-Uhlenbeck Process). The Ornstein-Uhlenbeckprocess has

i = EXti = 0

withi,i+1 = exp{|ti ti+1|/2}

and i,i = 1. So, our updating formula is

Xti+1 = exp{|ti ti+1|/2}Xti +(

1 exp{|ti ti+1|})Z.

Listing 3.9: Matlab code

1 alpha = 50; m = 10^4;

2

3 X = zeros(m,1); h = 1/m;

4 X(1) = randn;

5

6 for i = 1:m-1

7 X(i+1) = exp(-alpha * h / 2)*X(i)...

8 + sqrt(1 - exp(-alpha * h))*randn;

9 end

10

11 plot(h:h:1,X);

3.2 Brownian Motion

One of the most fundamental stochastic processes. It is a Gaussian pro-cess, a Markov process, a Levy process, a Martingale and a process that isclosely linked to the study of harmonic functions (do not worry if you do notknow all these terms). It can be used as a building block when consideringmany more complicated processes. For this reason, we will consider it inmuch more depth than any other Gaussian process.

Definition 3.2.1 (Brownian Motion). A stochastic process {Wt}t0 is calledBrownian motion (a Wiener process) if:

(i) It has independent increments.

(ii) It has stationary increments.

(iii) Wt N(0, t) for all t 0.

Introduction to Random Processes 117

Property 3.4 Ws trajectory, even if it is continuous, varies in a very irregular way thatmakes it non differentiable. Heuristically, this result comes from !W(t) = W(t + !t) W(t)being proportional to

!t . This property is often called Levys oscillatory property.

Property 3.5 The autocorrelation function of W(t) is given by:

RW W (t1, t2) = min(t1, t2). (7.47)

Proof: Indeed,

RW W (t1, t2) = E [W (t1)W (t2)] . (7.48)

Without loss of generality, we suppose t1 < t2 then

RW W (t1, t2) = E[(W (t2) W (t1)) W (t1) + W (t1)2

]. (7.49)

Since W(t) has independent increments,

E [(W (t2) W (t1)) W (t1)] = E [W (t1)] E [W (t2) W (t1)] = 0. (7.50)

Which implies that

RW W (t1, t2) = E[W (t1)2

]= Var(W (t1)) = t1 = min(t1, t2). (7.51)

Property 3.6 The standard Wiener process exhibits a very important property called themartingale property, that is:

E [W (tn)|W (tn1), W (tn2), . . . , W (t1)] = W (tn1), t1 < t2 < . . . < tn. (7.52)

Proof: Since W(t) has independent increments, we have:

E [W (tn)|W (tn1), . . . , W (t1)] (7.53)

or

E [W (tn) W (tn1) + W (tn1)|W (tn1), . . . , W (t1)] . (7.54)

Moreover, the expression

E [W (tn) W (tn1)|W (tn1), . . . , W (t1)] (7.55)

can be rewritten as

E [W (tn) W (tn1)|W (tn1)] = E [W (tn)|W (tn1)]E [W (tn1)|W (tn1)] . (7.56)

Processi stocastici // Classi di processi

Moto Browniano: un processo Gaussiano Markoviano

3.1. GAUSSIAN PROCESSES 107

andi = EXti .

By theorem 3.1.24, we know that (Xti , Xti+1) has a multivariate normal

distribution. In particular,(

XtiXti+1

) N

((ii+1

),

(i,i i,i+1i,i+1 i+1,i+1

)).

Using theorem 3.1.25, we have

Xti+1 |Xti = xi N(i +

i,i+1i, i

(xi i) , i+1,i+1 2i,i+1i,i

).

Algorithm 3.1.2 (Generating a Markovian Gaussian Process).

(i) Draw Z N(0, 1). Set Xt1 = 1 +i,iZ.

(ii) For i = 1, . . . ,m 1 draw Z N(0, 1) and set

Xti+1 = i +i,i+1i, i

(Xti i) +

i+1,i+1 2i,i+1i,i

Z.

Example 3.1.27 (Brownian Motion). Consider Brownian motion, which isa Markov process. Now,

i = EXti = 0

andi,i+1 = min(ti, ti+1) = ti.

So, our updating formula is

Xti+1 = Xti +(

ti+1 ti)Z.

Listing 3.8: Matlab code

1 m = 10^4; X = zeros(m,1);

2 h = 1/m;

3 X(1) = sqrt(h)*randn;

4

5 for i = 1:m-1

6 X(i+1) = X(i) + sqrt(h) * randn;

7 end

8

9 plot(h:h:1,X);

3.1. GAUSSIAN PROCESSES 107

andi = EXti .

By theorem 3.1.24, we know that (Xti , Xti+1) has a multivariate normal

distribution. In particular,(

XtiXti+1

) N

((ii+1

),

(i,i i,i+1i,i+1 i+1,i+1

)).

Using theorem 3.1.25, we have

Xti+1 |Xti = xi N(i +

i,i+1i, i

(xi i) , i+1,i+1 2i,i+1i,i

).

Algorithm 3.1.2 (Generating a Markovian Gaussian Process).

(i) Draw Z N(0, 1). Set Xt1 = 1 +i,iZ.

(ii) For i = 1, . . . ,m 1 draw Z N(0, 1) and set

Xti+1 = i +i,i+1i, i

(Xti i) +

i+1,i+1 2i,i+1i,i

Z.

Example 3.1.27 (Brownian Motion). Consider Brownian motion, which isa Markov process. Now,

i = EXti = 0

andi,i+1 = min(ti, ti+1) = ti.

So, our updating formula is

Xti+1 = Xti +(

ti+1 ti)Z.

Listing 3.8: Matlab code

1 m = 10^4; X = zeros(m,1);

2 h = 1/m;

3 X(1) = sqrt(h)*randn;

4

5 for i = 1:m-1

6 X(i+1) = X(i) + sqrt(h) * randn;

7 end

8

9 plot(h:h:1,X);

3.1. GAUSSIAN PROCESSES 107

andi = EXti .

By theorem 3.1.24, we know that (Xti , Xti+1) has a multivariate normal

distribution. In particular,(

XtiXti+1

) N

((ii+1

),

(i,i i,i+1i,i+1 i+1,i+1

)).

Using theorem 3.1.25, we have

Xti+1 |Xti = xi N(i +

i,i+1i, i

(xi i) , i+1,i+1 2i,i+1i,i

).

Algorithm 3.1.2 (Generating a Markovian Gaussian Process).

(i) Draw Z N(0, 1). Set Xt1 = 1 +i,iZ.

(ii) For i = 1, . . . ,m 1 draw Z N(0, 1) and set

Xti+1 = i +i,i+1i, i

(Xti i) +

i+1,i+1 2i,i+1i,i

Z.

Example 3.1.27 (Brownian Motion). Consider Brownian motion, which isa Markov process. Now,

i = EXti = 0

andi,i+1 = min(ti, ti+1) = ti.

So, our updating formula is

Xti+1 = Xti +(

ti+1 ti)Z.

Listing 3.8: Matlab code

1 m = 10^4; X = zeros(m,1);

2 h = 1/m;

3 X(1) = sqrt(h)*randn;

4

5 for i = 1:m-1

6 X(i+1) = X(i) + sqrt(h) * randn;

7 end

8

9 plot(h:h:1,X);

x1 x2 x3 x4

t1 t2 t3 t4

P(xt | xt-1 ) = Gaussiana

Processi stocastici // Simulazione di processi Gaussiani Matrice definita positiva: una matrice A (n x n) PD se e solo se

Se anche simmetrica, allora simmetrica definita positiva (SPD)

E semidefinita positiva (SPSD) se e solo se

Nota bene: le matrici di covarianza sono SPSD (e viceversa)

3.1. GAUSSIAN PROCESSES 91

INDEPENDENCEThere is nothing too difficult about dealing with the mean vector . How-

ever, the covariance matrix makes things pretty difficult, especially when wewish to simulate a high dimensional random vector. In order to simulateGaussian processes effectively, we need to exploit as many properties of co-variance matrices as possible.

Symmetric Positive Definite and Semi-Positive Definite Matrices

Covariance matrices are members of a family of matrices called symmetricpositive definite matrices.

Definition 3.1.6 (Positive Definite Matrices (Real-Valued)). An nn real-valued matrix, A, is positive definite if and only if

xAx > 0 for all x = 0.

If A is also symmetric, then A is called symmetric positive definite (SPD).Some important properties of SPD matrices are

(i) rank(A) = n.

(ii) |A| > 0.

(iii) Ai,i > 0.

(iv) A1 is SPD.

Lemma 3.1.7 (Necessary and sufficient conditions for an SPD). The fol-lowing are necessary and sufficient conditions for an n n matrix A to beSPD

(i) All the eigenvalues 1, . . . ,n of A are strictly positive.

(ii) There exists a unique matrix C such that A = CC, where C is a real-valued lower-triangular matrix with positive diagonal entries. This iscalled the Cholesky decomposition of A.

Definition 3.1.8 (Positive Semi-definite Matrices (Real-Valued)). An nnreal-valued matrix, A, is positive semi-definite if and only if

xAx 0 for all x = 0.

3.1. GAUSSIAN PROCESSES 91

INDEPENDENCEThere is nothing too difficult about dealing with the mean vector . How-

ever, the covariance matrix makes things pretty difficult, especially when wewish to simulate a high dimensional random vector. In order to simulateGaussian processes effectively, we need to exploit as many properties of co-variance matrices as possible.

Symmetric Positive Definite and Semi-Positive Definite Matrices

Covariance matrices are members of a family of matrices called symmetricpositive definite matrices.

Definition 3.1.6 (Positive Definite Matrices (Real-Valued)). An nn real-valued matrix, A, is positive definite if and only if

xAx > 0 for all x = 0.

If A is also symmetric, then A is called symmetric positive definite (SPD).Some important properties of SPD matrices are

(i) rank(A) = n.

(ii) |A| > 0.

(iii) Ai,i > 0.

(iv) A1 is SPD.

Lemma 3.1.7 (Necessary and sufficient conditions for an SPD). The fol-lowing are necessary and sufficient conditions for an n n matrix A to beSPD

(i) All the eigenvalues 1, . . . ,n of A are strictly positive.

(ii) There exists a unique matrix C such that A = CC, where C is a real-valued lower-triangular matrix with positive diagonal entries. This iscalled the Cholesky decomposition of A.

Definition 3.1.8 (Positive Semi-definite Matrices (Real-Valued)). An nnreal-valued matrix, A, is positive semi-definite if and only if

xAx 0 for all x = 0.

3.1. GAUSSIAN PROCESSES 91

INDEPENDENCEThere is nothing too difficult about dealing with the mean vector . How-

ever, the covariance matrix makes things pretty difficult, especially when wewish to simulate a high dimensional random vector. In order to simulateGaussian processes effectively, we need to exploit as many properties of co-variance matrices as possible.

Symmetric Positive Definite and Semi-Positive Definite Matrices

Covariance matrices are members of a family of matrices called symmetricpositive definite matrices.

Definition 3.1.6 (Positive Definite Matrices (Real-Valued)). An nn real-valued matrix, A, is positive definite if and only if

xAx > 0 for all x = 0.

If A is also symmetric, then A is called symmetric positive definite (SPD).Some important properties of SPD matrices are

(i) rank(A) = n.

(ii) |A| > 0.

(iii) Ai,i > 0.

(iv) A1 is SPD.

Lemma 3.1.7 (Necessary and sufficient conditions for an SPD). The fol-lowing are necessary and sufficient conditions for an n n matrix A to beSPD

(i) All the eigenvalues 1, . . . ,n of A are strictly positive.

(ii) There exists a unique matrix C such that A = CC, where C is a real-valued lower-triangular matrix with positive diagonal entries. This iscalled the Cholesky decomposition of A.

Definition 3.1.8 (Positive Semi-definite Matrices (Real-Valued)). An nnreal-valued matrix, A, is positive semi-definite if and only if

xAx 0 for all x = 0.

Decomposizione di Cholesky

Simulazione : utilizziamo la decomposizione di Cholesky

Processi stocastici // Simulazione di processi Gaussiani

Decomposizione di Cholesky

3.1. GAUSSIAN PROCESSES 93

Simulating Multivariate Normals

One possible way to simulate a multivariate normal random vector isusing the Cholesky decomposition.

Lemma 3.1.11. If Z N(0, I), = AA is a covariance matrix, andX = + AZ, then X N(,).

Proof. We know from lemma 3.1.4 that AZ is multivariate normal and sois + AZ. Because a multivariate normal random vector is completelydescribed by its mean vector and covariance matrix, we simple need to findEX and Var(X). Now,

EX = E+ EAZ = + A0 =

and

Var(X) = Var(+ AZ) = Var(AZ) = AVar(Z)A = AIA = AA = .

Thus, we can simulate a random vector X N(,) by

(i) Finding A such that = AA.

(ii) Generating Z N(0, I).

(iii) Returning X = + AZ.

Matlab makes things a bit confusing because its function chol producesa decomposition of the form BB. That is, A = B. It is important tobe careful and think about whether you want to generate a row or columnvector when generating multivariate normals.

The following code produces column vectors.

Listing 3.1: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + A * randn(3,1);

The following code produces row vectors.

Listing 3.2: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + randn(1,3) * A;

3.1. GAUSSIAN PROCESSES 93

Simulating Multivariate Normals

One possible way to simulate a multivariate normal random vector isusing the Cholesky decomposition.

Lemma 3.1.11. If Z N(0, I), = AA is a covariance matrix, andX = + AZ, then X N(,).

Proof. We know from lemma 3.1.4 that AZ is multivariate normal and sois + AZ. Because a multivariate normal random vector is completelydescribed by its mean vector and covariance matrix, we simple need to findEX and Var(X). Now,

EX = E+ EAZ = + A0 =

and

Var(X) = Var(+ AZ) = Var(AZ) = AVar(Z)A = AIA = AA = .

Thus, we can simulate a random vector X N(,) by

(i) Finding A such that = AA.

(ii) Generating Z N(0, I).

(iii) Returning X = + AZ.

Matlab makes things a bit confusing because its function chol producesa decomposition of the form BB. That is, A = B. It is important tobe careful and think about whether you want to generate a row or columnvector when generating multivariate normals.

The following code produces column vectors.

Listing 3.1: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + A * randn(3,1);

The following code produces row vectors.

Listing 3.2: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + randn(1,3) * A;

1. Trovare A tale che

2.

3.

Processi stocastici // Simulazione di processi Gaussiani

3.1. GAUSSIAN PROCESSES 93

Simulating Multivariate Normals

One possible way to simulate a multivariate normal random vector isusing the Cholesky decomposition.

Lemma 3.1.11. If Z N(0, I), = AA is a covariance matrix, andX = + AZ, then X N(,).

Proof. We know from lemma 3.1.4 that AZ is multivariate normal and sois + AZ. Because a multivariate normal random vector is completelydescribed by its mean vector and covariance matrix, we simple need to findEX and Var(X). Now,

EX = E+ EAZ = + A0 =

and

Var(X) = Var(+ AZ) = Var(AZ) = AVar(Z)A = AIA = AA = .

Thus, we can simulate a random vector X N(,) by

(i) Finding A such that = AA.

(ii) Generating Z N(0, I).

(iii) Returning X = + AZ.

Matlab makes things a bit confusing because its function chol producesa decomposition of the form BB. That is, A = B. It is important tobe careful and think about whether you want to generate a row or columnvector when generating multivariate normals.

The following code produces column vectors.

Listing 3.1: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + A * randn(3,1);

The following code produces row vectors.

Listing 3.2: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + randn(1,3) * A;

3.1. GAUSSIAN PROCESSES 93

Simulating Multivariate Normals

One possible way to simulate a multivariate normal random vector isusing the Cholesky decomposition.

Lemma 3.1.11. If Z N(0, I), = AA is a covariance matrix, andX = + AZ, then X N(,).

Proof. We know from lemma 3.1.4 that AZ is multivariate normal and sois + AZ. Because a multivariate normal random vector is completelydescribed by its mean vector and covariance matrix, we simple need to findEX and Var(X). Now,

EX = E+ EAZ = + A0 =

and

Var(X) = Var(+ AZ) = Var(AZ) = AVar(Z)A = AIA = AA = .

Thus, we can simulate a random vector X N(,) by

(i) Finding A such that = AA.

(ii) Generating Z N(0, I).

(iii) Returning X = + AZ.

Matlab makes things a bit confusing because its function chol producesa decomposition of the form BB. That is, A = B. It is important tobe careful and think about whether you want to generate a row or columnvector when generating multivariate normals.

The following code produces column vectors.

Listing 3.1: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + A * randn(3,1);

The following code produces row vectors.

Listing 3.2: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + randn(1,3) * A;

3.1. GAUSSIAN PROCESSES 93

Simulating Multivariate Normals

One possible way to simulate a multivariate normal random vector isusing the Cholesky decomposition.

Lemma 3.1.11. If Z N(0, I), = AA is a covariance matrix, andX = + AZ, then X N(,).

Proof. We know from lemma 3.1.4 that AZ is multivariate normal and sois + AZ. Because a multivariate normal random vector is completelydescribed by its mean vector and covariance matrix, we simple need to findEX and Var(X). Now,

EX = E+ EAZ = + A0 =

and

Var(X) = Var(+ AZ) = Var(AZ) = AVar(Z)A = AIA = AA = .

Thus, we can simulate a random vector X N(,) by

(i) Finding A such that = AA.

(ii) Generating Z N(0, I).

(iii) Returning X = + AZ.

Matlab makes things a bit confusing because its function chol producesa decomposition of the form BB. That is, A = B. It is important tobe careful and think about whether you want to generate a row or columnvector when generating multivariate normals.

The following code produces column vectors.

Listing 3.1: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + A * randn(3,1);

The following code produces row vectors.

Listing 3.2: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + randn(1,3) * A;

3.1. GAUSSIAN PROCESSES 93

Simulating Multivariate Normals

One possible way to simulate a multivariate normal random vector isusing the Cholesky decomposition.

Lemma 3.1.11. If Z N(0, I), = AA is a covariance matrix, andX = + AZ, then X N(,).

Proof. We know from lemma 3.1.4 that AZ is multivariate normal and sois + AZ. Because a multivariate normal random vector is completelydescribed by its mean vector and covariance matrix, we simple need to findEX and Var(X). Now,

EX = E+ EAZ = + A0 =

and

Var(X) = Var(+ AZ) = Var(AZ) = AVar(Z)A = AIA = AA = .

Thus, we can simulate a random vector X N(,) by

(i) Finding A such that = AA.

(ii) Generating Z N(0, I).

(iii) Returning X = + AZ.

Matlab makes things a bit confusing because its function chol producesa decomposition of the form BB. That is, A = B. It is important tobe careful and think about whether you want to generate a row or columnvector when generating multivariate normals.

The following code produces column vectors.

Listing 3.1: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + A * randn(3,1);

The following code produces row vectors.

Listing 3.2: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + randn(1,3) * A;

3.1. GAUSSIAN PROCESSES 93

Simulating Multivariate Normals

One possible way to simulate a multivariate normal random vector isusing the Cholesky decomposition.

Lemma 3.1.11. If Z N(0, I), = AA is a covariance matrix, andX = + AZ, then X N(,).

Proof. We know from lemma 3.1.4 that AZ is multivariate normal and sois + AZ. Because a multivariate normal random vector is completelydescribed by its mean vector and covariance matrix, we simple need to findEX and Var(X). Now,

EX = E+ EAZ = + A0 =

and

Var(X) = Var(+ AZ) = Var(AZ) = AVar(Z)A = AIA = AA = .

Thus, we can simulate a random vector X N(,) by

(i) Finding A such that = AA.

(ii) Generating Z N(0, I).

(iii) Returning X = + AZ.

Matlab makes things a bit confusing because its function chol producesa decomposition of the form BB. That is, A = B. It is important tobe careful and think about whether you want to generate a row or columnvector when generating multivariate normals.

The following code produces column vectors.

Listing 3.1: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + A * randn(3,1);

The following code produces row vectors.

Listing 3.2: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + randn(1,3) * A;

Simulazione:

3.1. GAUSSIAN PROCESSES 93

Simulating Multivariate Normals

One possible way to simulate a multivariate normal random vector isusing the Cholesky decomposition.

Lemma 3.1.11. If Z N(0, I), = AA is a covariance matrix, andX = + AZ, then X N(,).

Proof. We know from lemma 3.1.4 that AZ is multivariate normal and sois + AZ. Because a multivariate normal random vector is completelydescribed by its mean vector and covariance matrix, we simple need to findEX and Var(X). Now,

EX = E+ EAZ = + A0 =

and

Var(X) = Var(+ AZ) = Var(AZ) = AVar(Z)A = AIA = AA = .

Thus, we can simulate a random vector X N(,) by

(i) Finding A such that = AA.

(ii) Generating Z N(0, I).

(iii) Returning X = + AZ.

Matlab makes things a bit confusing because its function chol producesa decomposition of the form BB. That is, A = B. It is important tobe careful and think about whether you want to generate a row or columnvector when generating multivariate normals.

The following code produces column vectors.

Listing 3.1: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + A * randn(3,1);

The following code produces row vectors.

Listing 3.2: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + randn(1,3) * A;

3.1. GAUSSIAN PROCESSES 93

Simulating Multivariate Normals

One possible way to simulate a multivariate normal random vector isusing the Cholesky decomposition.

Lemma 3.1.11. If Z N(0, I), = AA is a covariance matrix, andX = + AZ, then X N(,).

Proof. We know from lemma 3.1.4 that AZ is multivariate normal and sois + AZ. Because a multivariate normal random vector is completelydescribed by its mean vector and covariance matrix, we simple need to findEX and Var(X). Now,

EX = E+ EAZ = + A0 =

and

Var(X) = Var(+ AZ) = Var(AZ) = AVar(Z)A = AIA = AA = .

Thus, we can simulate a random vector X N(,) by

(i) Finding A such that = AA.

(ii) Generating Z N(0, I).

(iii) Returning X = + AZ.

Matlab makes things a bit confusing because its function chol producesa decomposition of the form BB. That is, A = B. It is important tobe careful and think about whether you want to generate a row or columnvector when generating multivariate normals.

The following code produces column vectors.

Listing 3.1: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + A * randn(3,1);

The following code produces row vectors.

Listing 3.2: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + randn(1,3) * A;

Genera vettori colonna:

Genera vettori riga:

3.1. GAUSSIAN PROCESSES 93

Simulating Multivariate Normals

One possible way to simulate a multivariate normal random vector isusing the Cholesky decomposition.

Lemma 3.1.11. If Z N(0, I), = AA is a covariance matrix, andX = + AZ, then X N(,).

Proof. We know from lemma 3.1.4 that AZ is multivariate normal and sois + AZ. Because a multivariate normal random vector is completelydescribed by its mean vector and covariance matrix, we simple need to findEX and Var(X). Now,

EX = E+ EAZ = + A0 =

and

Var(X) = Var(+ AZ) = Var(AZ) = AVar(Z)A = AIA = AA = .

Thus, we can simulate a random vector X N(,) by

(i) Finding A such that = AA.

(ii) Generating Z N(0, I).

(iii) Returning X = + AZ.

Matlab makes things a bit confusing because its function chol producesa decomposition of the form BB. That is, A = B. It is important tobe careful and think about whether you want to generate a row or columnvector when generating multivariate normals.

The following code produces column vectors.

Listing 3.1: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + A * randn(3,1);

The following code produces row vectors.

Listing 3.2: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + randn(1,3) * A;

3.1. GAUSSIAN PROCESSES 93

Simulating Multivariate Normals

One possible way to simulate a multivariate normal random vector isusing the Cholesky decomposition.

Lemma 3.1.11. If Z N(0, I), = AA is a covariance matrix, andX = + AZ, then X N(,).

Proof. We know from lemma 3.1.4 that AZ is multivariate normal and sois + AZ. Because a multivariate normal random vector is completelydescribed by its mean vector and covariance matrix, we simple need to findEX and Var(X). Now,

EX = E+ EAZ = + A0 =

and

Var(X) = Var(+ AZ) = Var(AZ) = AVar(Z)A = AIA = AA = .

Thus, we can simulate a random vector X N(,) by

(i) Finding A such that = AA.

(ii) Generating Z N(0, I).

(iii) Returning X = + AZ.

Matlab makes things a bit confusing because its function chol producesa decomposition of the form BB. That is, A = B. It is important tobe careful and think about whether you want to generate a row or columnvector when generating multivariate normals.

The following code produces column vectors.

Listing 3.1: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + A * randn(3,1);

The following code produces row vectors.

Listing 3.2: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + randn(1,3) * A;

Simulazione : utilizziamo la decomposizione di Cholesky

3.1. GAUSSIAN PROCESSES 93

Simulating Multivariate Normals

One possible way to simulate a multivariate normal random vector isusing the Cholesky decomposition.

Lemma 3.1.11. If Z N(0, I), = AA is a covariance matrix, andX = + AZ, then X N(,).

Proof. We know from lemma 3.1.4 that AZ is multivariate normal and sois + AZ. Because a multivariate normal random vector is completelydescribed by its mean vector and covariance matrix, we simple need to findEX and Var(X). Now,

EX = E+ EAZ = + A0 =

and

Var(X) = Var(+ AZ) = Var(AZ) = AVar(Z)A = AIA = AA = .

Thus, we can simulate a random vector X N(,) by

(i) Finding A such that = AA.

(ii) Generating Z N(0, I).

(iii) Returning X = + AZ.

Matlab makes things a bit confusing because its function chol producesa decomposition of the form BB. That is, A = B. It is important tobe careful and think about whether you want to generate a row or columnvector when generating multivariate normals.

The following code produces column vectors.

Listing 3.1: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + A * randn(3,1);

The following code produces row vectors.

Listing 3.2: Matlab code

1 mu = [1 2 3];

2 Sigma = [3 1 2; 1 2 1; 2 1 5];

3 A = chol(Sigma);

4 X = mu + randn(1,3) * A;

Processi stocastici // Simulazione di processi Gaussiani Simulazione di un moto Browniano:

94 CHAPTER 3. GAUSSIAN PROCESSES ETC.

The complexity of the Cholesky decomposition of an arbitrary real-valuedmatrix is O(n3) floating point operations.

3.1.2 Simulating a Gaussian Processes Version 1

Because Gaussian processes are defined to be processes where, for anychoice of times t1, . . . , tn, the random vector (Xt1 , . . . , Xtn) has a multivariatenormal density and a multivariate normal density is completely describe bya mean vector and a covariance matrix , the probability distribution ofGaussian process is completely described if we have a way to construct themean vector and covariance matrix for an arbitrary choice of t1, . . . , tn.

We can do this using an expectation function

(t) = EXt

and a covariance function

r(s, t) = Cov(Xs, Xt).

Using these, we can simulate the values of a Gaussian process at fixed timest1, . . . , tn by calculating where i = (t1) and , where i,j = r(ti, tj) andthen simulating a multivariate normal vector.

In the following examples, we simulate the Gaussian processes at evenlyspaced times t1 = 1/h, t2 = 2/h, . . . , tn = 1.

Example 3.1.12 (Brownian Motion). Brownian motion is a very importantstochastic process which is, in some sense, the continuous time continuousstate space analogue of a simple random walk. It has an expectation function(t) = 0 and a covariance function r(s, t) = min(s, t).

Listing 3.3: Matlab code

1 %use mesh size h

2 m = 1000; h = 1/m; t = h : h : 1;

3 n = length(t);

4

5 %Make the mean vector

6 mu = zeros(1,n);

7 for i = 1:n

8 mu(i) = 0;

9 end

10

11 %Make the covariance matrix

12 Sigma = zeros(n,n);

94 CHAPTER 3. GAUSSIAN PROCESSES ETC.

The complexity of the Cholesky decomposition of an arbitrary real-valuedmatrix is O(n3) floating point operations.

3.1.2 Simulating a Gaussian Processes Version 1

Because Gaussian processes are defined to be processes where, for anychoice of times t1, . . . , tn, the random vector (Xt1 , . . . , Xtn) has a multivariatenormal density and a multivariate normal density is completely describe bya mean vector and a covariance matrix , the probability distribution ofGaussian process is completely described if we have a way to construct themean vector and covariance matrix for an arbitrary choice of t1, . . . , tn.

We can do this using an expectation function

(t) = EXt

and a covariance function

r(s, t) = Cov(Xs, Xt).

Using these, we can simulate the values of a Gaussian process at fixed timest1, . . . , tn by calculating where i = (t1) and , where i,j = r(ti, tj) andthen simulating a multivariate normal vector.

In the following examples, we simulate the Gaussian processes at evenlyspaced times t1 = 1/h, t2 = 2/h, . . . , tn = 1.

Example 3.1.12 (Brownian Motion). Brownian motion is a very importantstochastic process which is, in some sense, the continuous time continuousstate space analogue of a simple random walk. It has an expectation function(t) = 0 and a covariance function r(s, t) = min(s, t).

Listing 3.3: Matlab code

1 %use mesh size h

2 m = 1000; h = 1/m; t = h : h : 1;

3 n = length(t);

4

5 %Make the mean vector

6 mu = zeros(1,n);

7 for i = 1:n

8 mu(i) = 0;

9 end

10

11 %Make the covariance matrix

12 Sigma = zeros(n,n);

94 CHAPTER 3. GAUSSIAN PROCESSES ETC.

The complexity of the Cholesky decomposition of an arbitrary real-valuedmatrix is O(n3) floating point operations.

3.1.2 Simulating a Gaussian Processes Version 1

Because Gaussian processes are defined to be processes where, for anychoice of times t1, . . . , tn, the random vector (Xt1 , . . . , Xtn) has a multivariatenormal density and a multivariate normal density is completely describe bya mean vector and a covariance matrix , the probability distribution ofGaussian process is completely described if we have a way to construct themean vector and covariance matrix for an arbitrary choice of t1, . . . , tn.

We can do this using an expectation function

(t) = EXt

and a covariance function

r(s, t) = Cov(Xs, Xt).

Using these, we can simulate the values of a Gaussian process at fixed timest1, . . . , tn by calculating where i = (t1) and , where i,j = r(ti, tj) andthen simulating a multivariate normal vector.

In the following examples, we simulate the Gaussian processes at evenlyspaced times t1 = 1/h, t2 = 2/h, . . . , tn = 1.

Example 3.1.12 (Brownian Motion). Brownian motion is a very importantstochastic process which is, in some sense, the continuous time continuousstate space analogue of a simple random walk. It has an expectation function(t) = 0 and a covariance function r(s, t) = min(s, t).

Listing 3.3: Matlab code

1 %use mesh size h

2 m = 1000; h = 1/m; t = h : h : 1;

3 n = length(t);

4

5 %Make the mean vector

6 mu = zeros(1,n);

7 for i = 1:n

8 mu(i) = 0;

9 end

10

11 %Make the covariance matrix

12 Sigma = zeros(n,n);

94 CHAPTER 3. GAUSSIAN PROCESSES ETC.

The complexity of the Cholesky decomposition of an arbitrary real-valuedmatrix is O(n3) floating point operations.

3.1.2 Simulating a Gaussian Processes Version 1

Because Gaussian processes are defined to be processes where, for anychoice of times t1, . . . , tn, the random vector (Xt1 , . . . , Xtn) has a multivariatenormal density and a multivariate normal density is completely describe bya mean vector and a covariance matrix , the probability distribution ofGaussian process is completely described if we have a way to construct themean vector and covariance matrix for an arbitrary choice of t1, . . . , tn.

We can do this using an expectation function

(t) = EXt

and a covariance function

r(s, t) = Cov(Xs, Xt).

Using these, we can simulate the values of a Gaussian process at fixed timest1, . . . , tn by calculating where i = (t1) and , where i,j = r(ti, tj) andthen simulating a multivariate normal vector.

In the following examples, we simulate the Gaussian processes at evenlyspaced times t1 = 1/h, t2 = 2/h, . . . , tn = 1.

Example 3.1.12 (Brownian Motion). Brownian motion is a very importantstochastic process which is, in some sense, the continuous time continuousstate space analogue of a simple random walk. It has an expectation function(t) = 0 and a covariance function r(s, t) = min(s, t).

Listing 3.3: Matlab code

1 %use mesh size h

2 m = 1000; h = 1/m; t = h : h : 1;

3 n = length(t);

4

5 %Make the mean vector

6 mu = zeros(1,n);

7 for i = 1:n

8 mu(i) = 0;

9 end

10

11 %Make the covariance matrix

12 Sigma = zeros(n,n);

94 CHAPTER 3. GAUSSIAN PROCESSES ETC.

The complexity of the Cholesky decomposition of an arbitrary real-valuedmatrix is O(n3) floating point operations.

3.1.2 Simulating a Gaussian Processes Version 1

Because Gaussian processes are defined to be processes where, for anychoice of times t1, . . . , tn, the random vector (Xt1 , . . . , Xtn) has a multivariatenormal density and a multivariate normal density is completely describe bya mean vector and a covariance matrix , the probability distribution ofGaussian process is completely described if we have a way to construct themean vector and covariance matrix for an arbitrary choice of t1, . . . , tn.

We can do this using an expectation function

(t) = EXt

and a covariance function

r(s, t) = Cov(Xs, Xt).

Using these, we can simulate the values of a Gaussian process at fixed timest1, . . . , tn by calculating where i = (t1) and , where i,j = r(ti, tj) andthen simulating a multivariate normal vector.

In the following examples, we simulate the Gaussian processes at evenlyspaced times t1 = 1/h, t2 = 2/h, . . . , tn = 1.

Example 3.1.12 (Brownian Motion). Brownian motion is a very importantstochastic process which is, in some sense, the continuous time continuousstate space analogue of a simple random walk. It has an expectation function(t) = 0 and a covariance function r(s, t) = min(s, t).

Listing 3.3: Matlab code

1 %use mesh size h

2 m = 1000; h = 1/m; t = h : h : 1;

3 n = length(t);

4

5 %Make the mean vector

6 mu = zeros(1,n);

7 for i = 1:n

8 mu(i) = 0;

9 end

10

11 %Make the covariance matrix

12 Sigma = zeros(n,n);

94 CHAPTER 3. GAUSSIAN PROCESSES ETC.

The complexity of the Cholesky decomposition of an arbitrary real-valuedmatrix is O(n3) floating point operations.

3.1.2 Simulating a Gaussian Processes Version 1

Because Gaussian processes are defined to be processes where, for anychoice of times t1, . . . , tn, the random vector (Xt1 , . . . , Xtn) has a multivariatenormal density and a multivariate normal density is completely describe bya mean vector and a covariance matrix , the probability distribution ofGaussian process is completely described if we have a way to construct themean vector and covariance matrix for an arbitrary choice of t1, . . . , tn.

We can do this using an expectation function

(t) = EXt

and a covariance function

r(s, t) = Cov(Xs, Xt).

Using these, we can simulate the values of a Gaussian process at fixed timest1, . . . , tn by calculating where i = (t1) and , where i,j = r(ti, tj) andthen simulating a multivariate normal vector.

In the following examples, we simulate the Gaussian processes at evenlyspaced times t1 = 1/h, t2 = 2/h, . . . , tn = 1.

Example 3.1.12 (Brownian Motion). Brownian motion is a very importantstochastic process which is, in some sense, the continuous time continuousstate space analogue of a simple random walk. It has an expectation function(t) = 0 and a covariance function r(s, t) = min(s, t).

Listing 3.3: Matlab code

1 %use mesh size h

2 m = 1000; h = 1/m; t = h : h : 1;

3 n = length(t);

4

5 %Make the mean vector

6 mu = zeros(1,n);

7 for i = 1:n

8 mu(i) = 0;

9 end

10

11 %Make the covariance matrix

12 Sigma = zeros(n,n);

3.1. GAUSSIAN PROCESSES 95

13 for i = 1:n

14 for j = 1:n

15 Sigma(i,j) = min(t(i),t(j));

16 end

17 end

18

19 %Generate the multivariate normal vector

20 A = chol(Sigma);

21 X = mu + randn(1,n) * A;

22

23 %Plot

24 plot(t,X);

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 11.8

1.6

1.4

1.2

1

0.8

0.6

0.4

0.2

0

0.2

t

Xt

Figure 3.1.1: Plot of Brownain motion on [0, 1].

Example 3.1.13 (Ornstein-Uhlenbeck Process). Another very importantGaussian process is the Ornstein-Uhlenbeck process. This has expectationfunction (t) = 0 and covariance function r(s, t) = e|st|/2.

Listing 3.4: Matlab code

1 %use mesh size h

2 m = 1000; h = 1/m; t = h : h : 1;

3 n = length(t);

4 %paramter of OU process

5 alpha = 10;

6

94 CHAPTER 3. GAUSSIAN PROCESSES ETC.

The complexity of the Cholesky decomposition of an arbitrary real-valuedmatrix is O(n3) floating point operations.

3.1.2 Simulating a Gaussian Processes Version 1

Because Gaussian processes are defined to be processes where, for anychoice of times t1, . . . , tn, the random vector (Xt1 , . . . , Xtn) has a multivariatenormal density and a multivariate normal density is completely describe bya mean vector and a covariance matrix , the probability distribution ofGaussian process is completely described if we have a way to construct themean vector and covariance matrix for an arbitrary choice of t1, . . . , tn.

We can do this using an expectation function

(t) = EXt

and a covariance function

r(s, t) = Cov(Xs, Xt).

Using these, we can simulate the values of a Gaussian process at fixed timest1, . . . , tn by calculating where i = (t1) and , where i,j = r(ti, tj) andthen simulating a multivariate normal vector.

In the following examples, we simulate the Gaussian processes at evenlyspaced times t1 = 1/h, t2 = 2/h, . . . , tn = 1.

Example 3.1.12 (Brownian Motion). Brownian motion is a very importantstochastic process which is, in some sense, the continuous time continuousstate space analogue of a simple random walk. It has an expectation function(t) = 0 and a covariance function r(s, t) = min(s, t).

Listing 3.3: Matlab code

1 %use mesh size h

2 m = 1000; h = 1/m; t = h : h : 1;

3 n = length(t);

4

5 %Make the mean vector

6 mu = zeros(1,n);

7 for i = 1:n

8 mu(i) = 0;

9 end

10

11 %Make the covariance matrix

12 Sigma = zeros(n,n);

94 CHAPTER 3. GAUSSIAN PROCESSES ETC.

The complexity of the Cholesky decomposition of an arbitrary real-valuedmatrix is O(n3) floating point operations.

3.1.2 Simulating a Gaussian Processes Version 1

Because Gaussian processes are defined to be processes where, for anychoice of times t1, . . . , tn, the random vector (Xt1 , . . . , Xtn) has a multivariatenormal density and a multivariate normal density is completely describe bya mean vector and a covariance matrix , the probability distribution ofGaussian process is completely described if we have a way to construct themean vector and covariance matrix for an arbitrary choice of t1, . . . , tn.

We can do this using an expectation function

(t) = EXt

and a covariance function

r(s, t) = Cov(Xs, Xt).

Using these, we can simulate the values of a Gaussian process at fixed timest1, . . . , tn by calculating where i = (t1) and , where i,j = r(ti, tj) andthen simulating a multivariate normal vector.

In the following examples, we simulate the Gaussian processes at evenlyspaced times t1 = 1/h, t2 = 2/h, . . . , tn = 1.

Example 3.1.12 (Brownian Motion). Brownian motion is a very importantstochastic process which is, in some sense, the continuous time continuousstate space analogue of a simple random walk. It has an expectation function(t) = 0 and a covariance function r(s, t) = min(s, t).

Listing 3.3: Matlab code

1 %use mesh size h

2 m = 1000; h = 1/m; t = h : h : 1;

3 n = length(t);

4

5 %Make the mean vector

6 mu = zeros(1,n);

7 for i = 1:n

8 mu(i) = 0;

9 end

10

11 %Make the covariance matrix

12 Sigma = zeros(n,n);

94 CHAPTER 3. GAUSSIAN PROCESSES ETC.

The complexity of the Cholesky decomposition of an arbitrary real-valuedmatrix is O(n3) floating point operations.

3.1.2 Simulating a Gaussian Processes Version 1

Because Gaussian processes are defined to be processes where, for anychoice of times t1, . . . , tn, the random vector (Xt1 , . . . , Xtn) has a multivariatenormal density and a multivariate normal density is completely describe bya mean vector and a covariance matrix , the probability distribution ofGaussian process is completely described if we have a way to construct themean vector and covariance matrix for an arbitrary choice of t1, . . . , tn.

We can do this using an expectation function

(t) = EXt

and a covariance function

r(s, t) = Cov(Xs, Xt).

Using these, we can simulate the values of a Gaussian process at fixed timest1, . . . , tn by calculating where i = (t1) and , where i,j = r(ti, tj) andthen simulating a multivariate normal vector.

In the following examples, we simulate the Gaussian processes at evenlyspaced times t1 = 1/h, t2 = 2/h, . . . , tn = 1.

Example 3.1.12 (Brownian Motion). Brownian motion is a very importantstochastic process which is, in some sense, the continuous time continuousstate space analogue of a simple random walk. It has an expectation function(t) = 0 and a covariance function r(s, t) = min(s, t).

Listing 3.3: Matlab code

1 %use mesh size h

2 m = 1000; h = 1/m; t = h : h : 1;

3 n = length(t);

4

5 %Make the mean vector

6 mu = zeros(1,n);

7 for i = 1:n

8 mu(i) = 0;

9 end

10

11 %Make the covariance matrix

12 Sigma = zeros(n,n);

94 CHAPTER 3. GAUSSIAN PROCESSES ETC.

The complexity of the Cholesky decomposition of an arbitrary real-valuedmatrix is O(n3) floating point operations.

3.1.2 Simulating a Gaussian Processes Version 1

Because Gaussian processes are defined to be processes where, for anychoice of times t1, . . . , tn, the random vector (Xt1 , . . . , Xtn) has a multivariatenormal density and a multivariate normal density is completely describe bya mean vector and a covariance matrix , the probability distribution ofGaussian process is completely described if we have a way to construct themean vector and covariance matrix for an arbitrary choice of t1, . . . , tn.

We can do this using an expectation function

(t) = EXt

and a covariance function

r(s, t) = Cov(Xs, Xt).

Using these, we can simulate the values of a Gaussian process at fixed timest1, . . . , tn by calculating where i = (t1) and , where i,j = r(ti, tj) andthen simulating a multivariate normal vector.

In the following examples, we simulate the Gaussian processes at evenlyspaced times t1 = 1/h, t2 = 2/h, . . . , tn = 1.

Example 3.1.12 (Brownian Motion). Brownian motion is a very importantstochastic process which is, in some sense, the continuous time continuousstate space analogue of a simple random walk. It has an expectation function(t) = 0 and a covariance function r(s, t) = min(s, t).

Listing 3.3: Matlab code

1 %use mesh size h

2 m = 1000; h = 1/m; t = h : h : 1;

3 n = length(t);

4

5 %Make the mean vector

6 mu = zeros(1,n);

7 for i = 1:n

8 mu(i) = 0;

9 end

10

11 %Make the covariance matrix

12 Sigma = zeros(n,n);

Processi stocastici // Simulazione di processi Gaussiani Simulazione di un moto Browniano:

94 CHAPTER 3. GAUSSIAN PROCESSES ETC.

The complexity of the Cholesky decomposition of an arbitrary real-valuedmatrix is O(n3) floating point operations.

3.1.2 Simulating a Gaussian Processes Version 1

Because Gaussian processes are defined to be processes where, for anychoice of times t1, . . . , tn, the random vector (Xt1 , . . . , Xtn) has a multivariatenormal density and a multivariate normal density is completely describe bya mean vector and a covariance matrix , the probability distribution ofGaussian process is completely described if we have a way to construct themean vector and covariance matrix for an arbitrary choice of t1, . . . , tn.

We can do this using an expectation function

(t) = EXt

and a covariance function

r(s, t) = Cov(Xs, Xt).

Using these, we can simulate the values of a Gaussian process at fixed timest1, . . . , tn by calculating where i = (t1) and , where i,j = r(ti, tj) andthen simulating a multivariate normal vector.

In the following examples, we simulate the Gaussian processes at evenlyspaced times t1 = 1/h, t2 = 2/h, . . . , tn = 1.

Example 3.1.12 (Brownian Motion). Brownian motion is a very importantstochastic process which is, in some sense, the continuous time continuousstate space analogue of a simple random walk. It has an expectation function(t) = 0 and a covariance function r(s, t) = min(s, t).

Listing 3.3: Matlab code

1 %use mesh size h