applied stochastic processespavl/lec1_intro.pdf · this is a basic graduate course on stochastic...

APPLIED STOCHASTIC PROCESSES

G.A. Pavliotis

Department of Mathematics Imperial College London, UK

January 16, 2011

Pavliotis (IC) StochProc January 16, 2011 1 / 367

Lectures: Mondays, 10:00-12:00, Huxley 6M42.

Office Hours: By appointment.

Course webpage:http://www.ma.imperial.ac.uk/~pavl/stoch_proc.htm

Text: Lecture notes, available from the course webpage. Also,recommended reading from various textbooks/review articles.

The lecture notes are still in progress. Please send me yourcomments, suggestions and let me know of any typos/errorsthat you have spotted.


http://www.ma.imperial.ac.uk/~pavl/stoch_proc.htm

This is a basic graduate course on stochastic processes, aimedtowards PhD students in applied mathematics and theoreticalphysics.

The emphasis of the course will be on the presentation ofanalytical tools that are useful in the study of stochastic modelsthat appear in various problems in applied mathematics, physics,chemistry and biology.

The course will consist of three parts: Fundamentals of the theoryof stochastic processes, applications (reaction rate theory, surfacediffusion....) and non-equilibrium statistical mechanics.


1 PART I: FUNDAMENTALS OF CONTINUOUS TIMESTOCHASTIC PROCESSES

Elements of probability theory.Stochastic processes: basic definitions, examples.Continuous time Markov processes. Brownian motionDiffusion processes: basic definitions, the generator.Backward Kolmogorov and the Fokker–Planck (forwardKolmogorov) equations.Stochastic differential equations (SDEs); Itô calculus, Itô andStratonovich stochastic integrals, connection between SDEs andthe Fokker–Planck equation.Methods of solution for SDEs and for the Fokker-Planck equation.Ergodic properties and convergence to equilibrium.


2. PART II: APPLICATIONS.Asymptotic problems for the Fokker–Planck equation: overdamped(Smoluchowski) and underdamped (Freidlin-Wentzell) limits.Bistable stochastic systems: escape over a potential barrier, meanfirst passage time, calculation of escape rates etc.Brownian motion in a periodic potential. Stochastic models ofmolecular motors.Multiscale problems: averaging and homogenization.


3. PART III: NON-EQUILIBRIUM STATISTICAL MECHANICS.Derivation of stochastic differential equations from deterministicdynamics (heat bath models, projection operator techniques etc.).The fluctuation-dissipation theorem.Linear response theory.Derivation of macroscopic equations (hydrodynamics) andcalculation of transport coefficients.

4. ADDITIONAL TOPICS (time permitting): numerical methods,stochastic PDES, Markov chain Monte Carlo (MCMC)


Prerequisites

Basic knowledge of ODEs and PDEs.

Elementary probability theory.

Some familiarity with the theory of stochastic processes.


Lecture notes will be provided for all the material that we will coverin this course. They will be posted on the course webpage.

There are many excellent textbooks/review articles on appliedstochastic processes, at a level and style similar to that of thiscourse.Standard textbooks are

Gardiner: Handbook of stochastic methods (1985).Van Kampen: Stochastic processes in physics and chemistry(1981).Horsthemke and Lefever: Noise induced transitions (1984).Risken: The Fokker-Planck equation (1989).Oksendal: Stochastic differential equations (2003).Mazo: Brownian motion: fluctuations, dynamics and fluctuations(2002).Bhatthacharya and Waymire, Stochastic Processes andApplications (1990).


Other standard textbooks areNelson: Dynamical theories of Brownian motion (1967). Availablefrom the web (includes a very interesting historical overview of thetheory of Brownian motion).Chorin and Hald: Stochastic tools in mathematics and science(2006).Swanzing: Non-equilibrium statistical mechanics (2001).

The material on multiscale methods for stochastic processes,together with some of the introductory material, will be taken from

Pavliotis and Stuart: Multiscale methods: averaging andhomogenization (2008).


Excellent books on the mathematical theory of stochasticprocesses are

Karatzas and Shreeve: Brownian motion and stochastic calculus(1991).Revuz and Yor: Continuous martingales and Brownian motion(1999).Stroock: Probability theory, an analytic view (1993).


Well known review articles (available from the web) are:Chandrasekhar: Stochastic problems in physics and astronomy(1943).Hanggi and Thomas: Stochastic processes: time evolution,symmetries and linear response (1982).Hanggi, Talkner and Borkovec: Reaction rate theory: fifty yearsafter Kramers (1990).


The theory of stochastic processes started with Einstein’s work onthe theory of Brownian motion: Concerning the motion, asrequired by the molecular-kinetic theory of heat, of particlessuspended in liquids at rest (1905).

explanation of Brown’s observation (1827): when suspended inwater, small pollen grains are found to be in a very animated andirregular state of motion.Einstein’s theory is based on

A Markov chain model for the motion of the particle (molecule, pollengrain...).The idea that it makes more sense to talk about the probability offinding the particle at position x at time t , rather than about individualtrajectories.


In his work many of the main aspects of the modern theory ofstochastic processes can be found:

The assumption of Markovianity (no memory) expressed throughthe Chapman-Kolmogorov equation.The Fokker–Planck equation (in this case, the diffusion equation).The derivation of the Fokker-Planck equation from the master(Chapman-Kolmogorov) equation through a Kramers-Moyalexpansion.The calculation of a transport coefficient (the diffusion equation)using macroscopic (kinetic theory-based) considerations:

D =kBT6πηa

.

kB is Boltzmann’s constant, T is the temperature, η is the viscosityof the fluid and a is the diameter of the particle.


Einstein’s theory is based on the Fokker-Planck equation .Langevin (1908) developed a theory based on a stochasticdifferential equation . The equation of motion for a Brownianparticle is

md2xdt2 = −6πηa

dxdt

+ ξ,

where ξ is a random force.

There is complete agreement between Einstein’s theory andLangevin’s theory.

The theory of Brownian motion was developed independently bySmoluchowski.


The approaches of Langevin and Einstein represent the two mainapproaches in the theory of stochastic processes:

Study individual trajectories of Brownian particles. Their evolution isgoverned by a stochastic differential equation:

dXdt

= F (X) + Σ(X)ξ(t),

where ξ(t) is a random force.

Study the probability ρ(x , t) of finding a particle at position x attime t . This probability distribution satisfies the Fokker-Planckequation:

∂ρ

∂t= −∇ · (F (x)ρ) +

12∇∇ : (A(x)ρ),

where A(x) = Σ(x)Σ(x)T .


The theory of stochastic processes was developed during the 20thcentury:

Physics:Smoluchowksi.Planck (1917).Klein (1922).Ornstein and Uhlenbeck (1930).Kramers (1940).Chandrasekhar (1943).. . .

Mathematics:Wiener (1922).Kolmogorov (1931).Itô (1940’s).Doob (1940’s and 1950’s).. . .


The One-Dimensional Random Walk

We let time be discrete, i.e. t = 0, 1, . . . . Consider the followingstochastic process Sn:

S0 = 0;

at each time step it moves to ±1 with equal probability 12 .

In other words, at each time step we flip a fair coin. If the outcome isheads, we move one unit to the right. If the outcome is tails, we moveone unit to the left.Alternatively, we can think of the random walk as a sum of independentrandom variables:

Sn =n∑

j=1

Xj ,

where Xj ∈ −1,1 with P(Xj = ±1) = 12 .


We can simulate the random walk on a computer:

We need a (pseudo)random number generator to generate nindependent random variables which are uniformly distributed inthe interval [0,1].

If the value of the random variable is >12 then the particle moves

to the left, otherwise it moves to the right.

We then take the sum of all these random moves.

The sequence SnNn=1 indexed by the discrete time

T = 1, 2, . . .N is the path of the random walk. We use a linearinterpolation (i.e. connect the points n,Sn by straight lines) togenerate a continuous path .


0 5 10 15 20 25 30 35 40 45 50

−6

−4

−2

0

2

4

6

8

50−step random walk

Figure: Three paths of the random walk of length N = 50.


0 100 200 300 400 500 600 700 800 900 1000

−50

−40

−30

−20

−10

0

10

20

1000−step random walk

Figure: Three paths of the random walk of length N = 1000.


Every path of the random walk is different: it depends on theoutcome of a sequence of independent random experiments.

We can compute statistics by generating a large number of pathsand computing averages. For example, E(Sn) = 0, E(S2

n) = n.

The paths of the random walk (without the linear interpolation) arenot continuous: the random walk has a jump of size 1 at each timestep.

This is an example of a discrete time, discrete space stochasticprocesses.

The random walk is a time-homogeneous (the probabilistic lawof evolution is independent of time) Markov (the future dependsonly on the present and not on the past) process.

If we take a large number of steps, the random walk starts lookinglike a continuous time process with continuous paths.


Consider the sequence of continuous time stochastic processes

Z nt :=

1√n

Snt .

In the limit as n → ∞, the sequence Z nt converges (in some

appropriate sense) to a Brownian motion with diffusioncoefficient D = ∆x2

2∆t = 12 .


0 0.2 0.4 0.6 0.8 1−1.5

−1

−0.5

0

0.5

1

1.5

2

t

U(t)

mean of 1000 paths5 individual paths

Figure: Sample Brownian paths.


Brownian motion W (t) is a continuous time stochastic processeswith continuous paths that starts at 0 (W (0) = 0) and hasindependent, normally. distributed Gaussian increments.

We can simulate the Brownian motion on a computer using arandom number generator that generates normally distributed,independent random variables.


We can write an equation for the evolution of the paths of aBrownian motion Xt with diffusion coefficient D starting at x:

dXt =√

2DdWt , X0 = x .

This is an example of a stochastic differential equation .

The probability of finding Xt at y at time t , given that it was at x attime t = 0, the transition probability density ρ(y , t) satisfies thePDE

∂ρ

∂t= D

∂2ρ

∂y2 , ρ(y ,0) = δ(y − x).

This is an example of the Fokker-Planck equation .

The connection between Brownian motion and the diffusionequation was made by Einstein in 1905.


Why introduce randomness in the description of physical systems?

To describe outcomes of a repeated set of experiments. Think oftossing a coin repeatedly or of throwing a dice.To describe a deterministic system for which we have incompleteinformation: we have imprecise knowledge of initial and boundaryconditions or of model parameters.

ODEs with random initial conditions are equivalent to stochasticprocesses that can be described using stochastic differentialequations.

To describe systems for which we are not confident about thevalidity of our mathematical model.


To describe a dynamical system exhibiting very complicatedbehavior (chaotic dynamical systems). Determinism versuspredictability.

To describe a high dimensional deterministic system using asimpler, low dimensional stochastic system. Think of the physicalmodel for Brownian motion (a heavy particle colliding with manysmall particles).

To describe a system that is inherently random. Think of quantummechanics.


ELEMENTS OF PROBABILITY THEORY


A collection of subsets of a set Ω is called a σ–algebra if itcontains Ω and is closed under the operations of takingcomplements and countable unions of its elements.

A sub-σ–algebra is a collection of subsets of a σ–algebra whichsatisfies the axioms of a σ–algebra.

A measurable space is a pair (Ω,F) where Ω is a set and F is aσ–algebra of subsets of Ω.

Let (Ω,F) and (E ,G) be two measurable spaces. A functionX : Ω 7→ E such that the event

ω ∈ Ω : X (ω) ∈ A =: X ∈ A

belongs to F for arbitrary A ∈ G is called a measurable function orrandom variable.


Let (Ω,F) be a measurable space. A function µ : F 7→ [0,1] iscalled a probability measure if µ(∅) = 1, µ(Ω) = 1 andµ(∪∞

k=1Ak ) =∑∞

k=1 µ(Ak) for all sequences of pairwise disjointsets Ak∞k=1 ∈ F .

The triplet (Ω,F , µ) is called a probability space.

Let X be a random variable (measurable function) from (Ω,F , µ)to (E ,G). If E is a metric space then we may define expectationwith respect to the measure µ by

E[X ] =

∫

ΩX (ω) dµ(ω).

More generally, let f : E 7→ R be G–measurable. Then,

E[f (X )] =

∫

Ωf (X (ω)) dµ(ω).


Let U be a topological space. We will use the notation B(U) todenote the Borel σ–algebra of U: the smallest σ–algebracontaining all open sets of U. Every random variable from aprobability space (Ω,F , µ) to a measurable space (E ,B(E))induces a probability measure on E :

µX (B) = PX−1(B) = µ(ω ∈ Ω; X (ω) ∈ B), B ∈ B(E).

The measure µX is called the distribution (or sometimes the law)of X .

Example

Let I denote a subset of the positive integers. A vectorρ0 = ρ0,i , i ∈ I is a distribution on I if it has nonnegative entries andits total mass equals 1:

∑i∈I ρ0,i = 1.


We can use the distribution of a random variable to computeexpectations and probabilities:

E[f (X )] =

∫

Sf (x) dµX (x)

and

P[X ∈ G] =

∫

GdµX (x), G ∈ B(E).

When E = Rd and we can write dµX (x) = ρ(x) dx , then we refer

to ρ(x) as the probability density function (pdf), or density withrespect to Lebesque measure for X .

When E = Rd then by Lp(Ω; Rd ), or sometimes Lp(Ω;µ) or even

simply Lp(µ), we mean the Banach space of measurable functionson Ω with norm

‖X‖Lp =(

E|X |p)1/p

.


Example

Consider the random variable X : Ω 7→ R with pdf

γσ,m(x) := (2πσ)−12 exp

(−(x − m)2

2σ

).

Such an X is termed a Gaussian or normal random variable. Themean is

EX =

∫

R

xγσ,m(x) dx = m

and the variance is

E(X − m)2 =

∫

R

(x − m)2γσ,m(x) dx = σ.


Example (Continued)

Let m ∈ Rd and Σ ∈ R

d×d be symmetric and positive definite. Therandom variable X : Ω 7→ R

d with pdf

γΣ,m(x) :=((2π)d detΣ

)− 12

exp(−1

2〈Σ−1(x − m), (x − m)〉

)

is termed a multivariate Gaussian or normal random variable.The mean is

E(X ) = m (1)

and the covariance matrix is

E

((X − m) ⊗ (X − m)

)= Σ. (2)


Since the mean and variance specify completely a Gaussianrandom variable on R, the Gaussian is commonly denoted byN (m, σ). The standard normal random variable is N (0,1).

Since the mean and covariance matrix completely specify aGaussian random variable on R

d , the Gaussian is commonlydenoted by N (m,Σ).


The Characteristic Function

Many of the properties of (sums of) random variables can bestudied using the Fourier transform of the distribution function.

The characteristic function of the random variable X is definedto be the Fourier transform of the distribution function

φ(t) =

∫

R

eitλ dµX (λ) = E(eitX ). (3)

The characteristic function determines uniquely the distributionfunction of the random variable, in the sense that there is aone-to-one correspondance between F (λ) and φ(t).

The characteristic function of a N (m,Σ) is

φ(t) = e〈m,t〉− 12 〈t,Σt〉.


Lemma

Let X1,X2, . . .Xn be independent random variables withcharacteristic functions φj(t), j = 1, . . . n and let Y =

∑nj=1 Xj with

characteristic function φY (t). Then

φY (t) = Πnj=1φj(t).

Lemma

Let X be a random variable with characteristic function φ(t) andassume that it has finite moments. Then

E(X k) =1ikφ(k)(0).


Types of Convergence and Limit Theorems

One of the most important aspects of the theory of randomvariables is the study of limit theorems for sums of randomvariables.

The most well known limit theorems in probability theory are thelaw of large numbers and the central limit theorem.

There are various different types of convergence for sequences orrandom variables.


DefinitionLet Zn∞n=1 be a sequence of random variables. We will say that

(a) Zn converges to Z with probability one (almost surely) if

P(

limn→+∞

Zn = Z)

= 1.

(b) Zn converges to Z in probability if for every ε > 0

limn→+∞

P(|Zn − Z | > ε

)= 0.

(c) Zn converges to Z in Lp if

limn→+∞

E[∣∣Zn − Z

∣∣p] = 0.

(d) Let Fn(λ),n = 1, · · · + ∞, F (λ) be the distribution functions ofZn n = 1, · · · + ∞ and Z , respectively. Then Zn converges to Z indistribution if

limn→+∞

Fn(λ) = F (λ)

for all λ ∈ R at which F is continuous.Pavliotis (IC) StochProc January 16, 2011 39 / 367

Let Xn∞n=1 be iid random variables with EXn = V . Then, thestrong law of large numbers states that average of the sum ofthe iid converges to V with probability one:

P

(

limn→+∞

1N

N∑

n=1

Xn = V

)

= 1.

The strong law of large numbers provides us with informationabout the behavior of a sum of random variables (or, a largenumber or repetitions of the same experiment) on average.We can also study fluctuations around the average behavior.Indeed, let E(Xn − V )2 = σ2. Define the centered iid randomvariables Yn = Xn − V . Then, the sequence of random variables

1σ√

N

∑Nn=1 Yn converges in distribution to a N (0,1) random

variable:

limn→+∞

P

(1

σ√

N

N∑

n=1

Yn 6 a

)=

∫ a

−∞

1√2π

e− 12 x2

dx .


Assume that E|X | <∞ and let G be a sub–σ–algebra of F . Theconditional expectation of X with respect to G is defined to bethe function E[X |G] : Ω 7→ E which is G–measurable and satisfies

∫

GE[X |G] dµ =

∫

GX dµ ∀G ∈ G.

We can define E[f (X )|G] and the conditional probabilityP[X ∈ F |G] = E[IF (X )|G], where IF is the indicator function of F , ina similar manner.


ELEMENTS OF THE THEORY OF STOCHASTIC PROCESSES


Let T be an ordered set. A stochastic process is a collection ofrandom variables X = Xt ; t ∈ T where, for each fixed t ∈ T , Xt

is a random variable from (Ω,F) to (E ,G).

The measurable space Ω,F is called the sample space . Thespace (E ,G) is called the state space .

In this course we will take the set T to be [0,+∞).

The state space E will usually be Rd equipped with the σ–algebra

of Borel sets.

A stochastic process X may be viewed as a function of both t ∈ Tand ω ∈ Ω. We will sometimes write X (t),X (t , ω) or Xt(ω) insteadof Xt . For a fixed sample point ω ∈ Ω, the function Xt(ω) : T 7→ Eis called a sample path (realization, trajectory) of the process X .


The finite dimensional distributions (fdd) of a stochasticprocess are the distributions of the Ek–valued random variables(X (t1),X (t2), . . . ,X (tk )) for arbitrary positive integer k andarbitrary times ti ∈ T , i ∈ 1, . . . , k:

F (x) = P(X (ti) 6 xi , i = 1, . . . , k)

with x = (x1, . . . , xk ).

We will say that two processes Xt and Yt are equivalent if theyhave same finite dimensional distributions.

From experiments or numerical simulations we can only obtaininformation about the (fdd) of a process.


Definition

A Gaussian process is a stochastic processes for which E = Rd and

all the finite dimensional distributions are Gaussian

F (x) = P(X (ti) 6 xi , i = 1, . . . , k)

= (2π)−n/2(detKk )−1/2 exp[−1

2〈K−1

k (x − µk ), x − µk 〉],

for some vector µk and a symmetric positive definite matrix Kk .

A Gaussian process x(t) is characterized by its mean

m(t) := Ex(t)

and the covariance function

C(t , s) = E

((x(t) − m(t)

)⊗(x(s) − m(s)

)).

Thus, the first two moments of a Gaussian process are sufficientfor a complete characterization of the process.


Let(Ω,F ,P

)be a probability space. Let Xt , t ∈ T (with T = R or Z) be

a real-valued random process on this probability space with finitesecond moment, E|Xt |2 < +∞ (i.e. Xt ∈ L2 ).

Definition

A stochastic process Xt ∈ L2 is called second-order stationary orwide-sense stationary if the first moment EXt is a constant and thesecond moment E(XtXs) depends only on the difference t − s:

EXt = µ, E(XtXs) = C(t − s).


The constant m is called the expectation of the process Xt . Wewill set m = 0.

The function C(t) is called the covariance or the autocorrelationfunction of the Xt .

Notice that C(t) = E(XtX0), whereas C(0) = E(X 2t ), which is

finite, by assumption.

Since we have assumed that Xt is a real valued process, we havethat C(t) = C(−t), t ∈ R.


Continuity properties of the covariance function are equivalent tocontinuity properties of the paths of Xt .

Lemma

Assume that the covariance function C(t) of a second order stationaryprocess is continuous at t = 0. Then it is continuous for all t ∈ R.Furthermore, the continuity of C(t) is equivalent to the continuity of theprocess Xt in the L2-sense.


Proof.Fix t ∈ R. We calculate:

|C(t + h) − C(t)|2 = |E(Xt+hX0) − E(XtX0)|2 = E|((Xt+h − Xt)X0)|26 E(X0)

2E(Xt+h − Xt)

2

= C(0)(EX 2t+h + EX 2

t − 2EXtXt+h)

= 2C(0)(C(0) − C(h)) → 0,

as h → 0.Assume now that C(t) is continuous. From the above calculation wehave

E|Xt+h − Xt |2 = 2(C(0) − C(h)), (4)

which converges to 0 as h → 0. Conversely, assume that Xt isL2-continuous. Then, from the above equation we getlimh→0 C(h) = C(0).


Notice that form (4) we immediately conclude thatC(0) > C(h), h ∈ R.

The Fourier transform of the covariance function of a second orderstationary process always exists. This enables us to study secondorder stationary processes using tools from Fourier analysis.

To make the link between second order stationary processes andFourier analysis we will use Bochner’s theorem, which applies toall nonnegative functions.

DefinitionA function f (x) : R 7→ R is called nonnegative definite if

n∑

i ,j=1

f (ti − tj)ci cj > 0 (5)

for all n ∈ N, t1, . . . tn ∈ R, c1, . . . cn ∈ C.


LemmaThe covariance function of second order stationary process is anonnegative definite function.

Proof.

We will use the notation X ct :=

∑ni=1 Xti ci . We have.

n∑

i ,j=1

C(ti − tj)ci cj =

n∑

i ,j=1

EXti Xtj ci cj

= E

n∑

i=1

Xti ci

n∑

j=1

Xtj cj

= E(X c

t X ct

)

= E|X ct |2 > 0.


Theorem(Bochner ) There is a one-to-one correspondence between the set ofcontinuous nonnegative definite functions and the set of finitemeasures on the Borel σ-algebra of R: if ρ is a finite measure, then

C(t) =

∫

R

eixt ρ(dx) (6)

in nonnegative definite. Conversely, any nonnegative definite functioncan be represented in the form (6).

DefinitionLet Xt be a second order stationary process with covariance C(t)whose Fourier transform is the measure ρ(dx). The measure ρ(dx) iscalled the spectral measure of the process Xt .


In the following we will assume that the spectral measure isabsolutely continuous with respect to the Lebesgue measure on R

with density f (x), dρ(x) = f (x)dx .

The Fourier transform f (x) of the covariance function is called thespectral density of the process:

f (x) =1

2π

∫ ∞

−∞e−itxC(t) dt .

From (6) it follows that that the covariance function of a meanzero, second order stationary process is given by the inverseFourier transform of the spectral density:

C(t) =

∫ ∞

−∞eitx f (x) dx .

In most cases, the experimentally measured quantity is thespectral density (or power spectrum) of the stochastic process.


The correlation function of a second order stationary processenables us to associate a time scale to Xt , the correlation timeτcor :

τcor =1

C(0)

∫ ∞

0C(τ) dτ =

∫ ∞

0E(XτX0)/E(X 2

0 ) dτ.

The slower the decay of the correlation function, the larger thecorrelation time is. We have to assume sufficiently fast decay ofcorrelations so that the correlation time is finite.


ExampleConsider the mean zero, second order stationary process withcovariance function

R(t) =Dα

e−α|t|. (7)

The spectral density of this process is:

f (x) =1

2πDα

∫ +∞

−∞e−ixte−α|t| dt

=1

2πDα

(∫ 0

−∞e−ixteαt dt +

∫ +∞

0e−ixte−αt dt

)

=1

2πDα

(1

ix + α+

1−ix + α

)

=Dπ

1x2 + α2 .


Example (Continued)This function is called the Cauchy or the Lorentz distribution.

The Gaussian stochastic process with covariance function (7) iscalled the stationary Ornstein-Uhlenbeck process .

The correlation time is (we have that C(0) = D/(α))

τcor =

∫ ∞

0e−αt dt = α−1.


The OU process was introduced by Ornstein and Uhlenbeck in 1930(G.E. Uhlenbeck, L.S. Ornstein, Phys. Rev. 36, 823 (1930)) as a modelfor the velocity of a Brownian particle. It is of interest to calculate thestatistics of the position of the Brownian particle, i.e. of the integral

X (t) =

∫ t

0Y (s) ds, (8)

where Y (t) denotes the stationary OU process.

LemmaLet Y (t) denote the stationary OU process with covariancefunction (7). Then the position process (8) is a mean zero Gaussianprocess with covariance function

E(X (t)X (s)) = 2 min(t , s) + e−min(t,s) + e−max(t,s) − e−|t−s| − 1.


Second order stationary processes are ergodic: time averages equalphase space (ensemble) averages. An example of an ergodic theoremfor a stationary processes is the following L2 (mean-square) ergodictheorem.

TheoremLet Xtt>0 be a second order stationary process on a probabilityspace Ω, F , P with mean µ and covariance R(t), and assume thatR(t) ∈ L1(0,+∞). Then

limT→+∞

E

∣∣∣∣∣1T

∫ T

0X (s) ds − µ

∣∣∣∣∣

2

= 0. (9)


Proof.We have

E

∣∣∣∣∣1T

∫ T

0X (s) ds − µ

∣∣∣∣∣

2

=1

T 2

∫ T

0

∫ T

0R(t − s) dtds

=2

T 2

∫ T

0

∫ t

0R(t − s) dsdt

=2

T 2

∫ T

0(T − v)R(u) du → 0,

using the dominated convergence theorem and the assumptionR(·) ∈ L1. In the above we used the fact that R is a symmetricfunction, together with the change of variables u = t − s, v = t and anintegration over v .


Assume that µ = 0. From the above calculation we can conclude that,under the assumption that R(·) decays sufficiently fast at infinity, fort ≫ 1 we have that

E

(∫ t

0X (t) dt

)2

≈ 2Dt ,

where

D =

∫ ∞

0R(t) dt

is the diffusion coefficient . Thus, one expects that at sufficiently longtimes and under appropriate assumptions on the correlation function,the time integral of a stationary process will approximate a Brownianmotion with diffusion coefficient D.This kind of analysis was initiated in G.I. Taylor, Diffusion byContinuous Movements Proc. London Math. Soc..1922; s2-20:196-212.


DefinitionA stochastic process is called (strictly) stationary if all finitedimensional distributions are invariant under time translation: for anyinteger k and times ti ∈ T , the distribution of (X (t1),X (t2), . . . ,X (tk )) isequal to that of (X (s + t1),X (s + t2), . . . ,X (s + tk )) for any s such thats + ti ∈ T for all i ∈ 1, . . . , k. In other words,

P(Xt1+t ∈ A1,Xt2+t ∈ A2 . . .Xtk +t ∈ Ak )

= P(Xt1 ∈ A1,Xt2 ∈ A2 . . .Xtk ∈ Ak ), ∀t ∈ T .

Let Xt be a strictly stationary stochastic process with finite secondmoment (i.e. Xt ∈ L2). The definition of strict stationarity impliesthat EXt = µ, a constant, and E((Xt − µ)(Xs − µ)) = C(t − s).Hence, a strictly stationary process with finite second moment isalso stationary in the wide sense. The converse is not true.


Remarks

1 A sequence Y0, Y1, . . . of independent, identically distributedrandom variables is a stationary process withR(k) = σ2δk0, σ

2 = E(Yk )2.2 Let Z be a single random variable with known distribution and set

Zj = Z , j = 0,1,2, . . . . Then the sequence Z0, Z1, Z2, . . . is astationary sequence with R(k) = σ2.

3 The first two moments of a Gaussian process are sufficient for acomplete characterization of the process. A corollary of this is thata second order stationary Gaussian process is also a (strictly)stationary process.


The most important continuous time stochastic process isBrownian motion . Brownian motion is a mean zero, continuous(i.e. it has continuous sample paths: for a.e ω ∈ Ω the function Xt

is a continuous function of time) process with independentGaussian increments.

A process Xt has independent increments if for every sequencet0 < t1...tn the random variables

Xt1 − Xt0 , Xt2 − Xt1 , . . . ,Xtn − Xtn−1

are independent.

If, furthermore, for any t1, t2 and Borel set B ⊂ R

P(Xt2+s − Xt1+s ∈ B)

is independent of s, then the process Xt has stationaryindependent increments .


Definition

A one dimensional standard Brownian motion W (t) : R+ → R is a

real valued stochastic process with the following properties:1 W (0) = 0;2 W (t) is continuous;3 W (t) has independent increments.4 For every t > s > 0 W (t) − W (s) has a Gaussian distribution with

mean 0 and variance t − s. That is, the density of the randomvariable W (t) − W (s) is

g(x ; t , s) =(

2π(t − s))− 1

2exp

(− x2

2(t − s)

); (10)


Definition (Continued)

A d–dimensional standard Brownian motion W (t) : R+ → R

d is acollection of d independent one dimensional Brownian motions:

W (t) = (W1(t), . . . ,Wd (t)),

where Wi(t), i = 1, . . . ,d are independent one dimensionalBrownian motions. The density of the Gaussian random vectorW (t) − W (s) is thus

g(x ; t , s) =(

2π(t − s))−d/2

exp(− ‖x‖2

2(t − s)

).

Brownian motion is sometimes referred to as the Wiener process .


0 0.2 0.4 0.6 0.8 1−1.5

−1

−0.5

0

0.5

1

1.5

2

t

U(t)

mean of 1000 paths5 individual paths

Figure: Brownian sample paths


It is possible to prove rigorously the existence of the Wiener process(Brownian motion):

Theorem(Wiener) There exists an almost-surely continuous process Wt withindependent increments such and W0 = 0, such that for each t > therandom variable Wt is N (0, t). Furthermore, Wt is almost surely locallyHölder continuous with exponent α for any α ∈ (0, 1

2).

Notice that Brownian paths are not differentiable.


Brownian motion is a Gaussian process. For the d–dimensionalBrownian motion, and for I the d × d dimensional identity, we have(see (1) and (2))

EW (t) = 0 ∀t > 0

andE

((W (t) − W (s)) ⊗ (W (t) − W (s))

)= (t − s)I. (11)

Moreover,E

(W (t) ⊗ W (s)

)= min(t , s)I. (12)


From the formula for the Gaussian density g(x , t − s), eqn. (10),we immediately conclude that W (t) − W (s) andW (t + u) − W (s + u) have the same pdf. Consequently, Brownianmotion has stationary increments.

Notice, however, that Brownian motion itself is not a stationaryprocess.

Since W (t) = W (t) − W (0), the pdf of W (t) is

g(x , t) =1√2πt

e−x2/2t .

We can easily calculate all moments of the Brownian motion:

E(xn(t)) =1√2πt

∫ +∞

−∞xne−x2/2t dx

= 1.3 . . . (n − 1)tn/2, n even,

0, n odd.


We can define the OU process through the Brownian motion via atime change.

LemmaLet W (t) be a standard Brownian motion and consider the process

V (t) = e−tW (e2t).

Then V (t) is a Gaussian second order stationary process with mean 0and covariance

K (s, t) = e−|t−s|.


Definition

A (normalized) fractional Brownian motion W Ht , t > 0 with Hurst

parameter H ∈ (0,1) is a centered Gaussian process with continuoussample paths whose covariance is given by

E(W Ht W H

s ) =12

(s2H + t2H − |t − s|2H). (13)

Fractional Brownian motion has the following properties.

1 When H = 12 , W

12

t becomes the standard Brownian motion.2 W H

0 = 0, EW Ht = 0, E(W H

t )2 = |t |2H , t > 0.3 It has stationary increments, E(W H

t − W Hs )2 = |t − s|2H .

4 It has the following self similarity property

(W Hαt , t > 0) = (αHW H

t , t > 0), α > 0,

where the equivalence is in law.


The Poisson Process

Another fundamental continuous time process is the Poissonprocess :

Definition

The Poisson process with intensity λ, denoted by N(t), is aninteger-valued, continuous time, stochastic process with independentincrements satisfying

P[(N(t) − N(s)) = k ] =e−λ(t−s)

(λ(t − s)

)k

k!, t > s > 0, k ∈ N.

Both Brownian motion and the Poisson process arehomogeneous (or time-homogeneous ): the incrementsbetween successive times s and t depend only on t − s.


A very useful result is that we can expand every centered(EXt = 0) stochastic process with continuous covariance (andhence, every L2-continuous centered stochastic process) into arandom Fourier series.

Let (Ω,F ,P) be a probability space and Xtt∈T a centeredprocess which is mean square continuous. Then, theKarhunen-Loéve theorem states that Xt can be expanded in theform

Xt =

∞∑

n=1

ξnφn(t), (14)

where the ξn are an orthogonal sequence of random variableswith E|ξk |2 = λk , where λk φn∞k=1 are the eigenvalues andeigenfunctions of the integral operator whose kernel is thecovariance of the processes Xt . The convergence is in L2(P) forevery t ∈ T .

If Xt is a Gaussian process then the ξk are independent Gaussianrandom variables.


Set T = [0,1].Let K : L2(T ) → L2(T ) be defined by

Kψ(t) =

∫ 1

0K (s, t)ψ(s) ds.

The kernel of this integral operator is a continuous (in both s andt), symmetric, nonnegative function. Hence, the correspondingintegral operator has eigenvalues, eigenfunctions

Kφn = λn, λn > 0, (φn, φm)L2 = δnm,

such that the covariance operator can be expanded in a uniformlyconvergent series

B(t , s) =

∞∑

n=1

λnφn(t)φn(s).


The random variables ξn are defined as

ξn =

∫ 1

0X (t)φn(t) dt .

The orthogonality of the eigenfunctions of the covariance operatorimplies that

E(ξnξm) = λnδnm.

Thus the random variables are orthogonal. When X (t) isGaussian, then ξk are Gaussian random variables. Furthermore,since they are also orthogonal, they are independent Gaussianrandom variables.


Example

The Karhunen-Loéve Expansion for Brownian Motion We setT = [0,1].The covariance function of Brownian motion isC(t , s) = min(t , s). The eigenvalue problem Cψn = λnψn becomes

∫ 1

0min(t , s)ψn(s) ds = λnψn(t).

Or, ∫ 1

0sψn(s) ds + t

∫ 1

tψn(s) ds = λnψn(t).

We differentiate this equation twice:

∫ 1

tψn(s) ds = λnψ

′n(t) and − ψn(t) = λnψ

′′n(t),

where primes denote differentiation with respect to t .


ExampleFrom the above equations we immediately see that the right boundaryconditions are ψ(0) = ψ′(1) = 0. The eigenvalues and eigenfunctionsare

ψn(t) =√

2 sin(

12

(2n + 1)πt), λn =

(2

(2n + 1)π

)2

.

Thus, the Karhunen-Loéve expansion of Brownian motion on [0,1] is

Wt =

√2π

∞∑

n=1

ξn2

(2n + 1)πsin(

12(2n + 1)πt

). (15)


The Path Space

Let (Ω,F , µ) be a probability space, (E , ρ) a metric space and letT = [0,∞). Let Xt be a stochastic process from (Ω,F , µ) to(E , ρ) with continuous sample paths.

The above means that for every ω ∈ Ω we have thatXt ∈ CE := C([0,∞); E).

The space of continuous functions CE is called the path space ofthe stochastic process.

We can put a metric on E as follows:

ρE (X 1,X 2) :=

∞∑

n=1

12n max

06t6nmin

(ρ(X 1

t ,X2t ),1

).

We can then define the Borel sets on CE , using the topologyinduced by this metric, and Xt can be thought of as a randomvariable on (Ω,F , µ) with state space (CE ,B(CE )).


The probability measure PX−1t on (CE ,B(CE)) is called the law of

Xt.

The law of a stochastic process is a probability measure on itspath space.

ExampleThe space of continuous functions CE is the path space of Brownianmotion (the Wiener process). The law of Brownian motion, that is themeasure that it induces on C([0,∞),Rd ), is known as the Wienermeasure .


MARKOV STOCHASTIC PROCESSES


Let (Ω,F) be a measurable space and T an ordered set. LetX = Xt(ω) be a stochastic process from the sample space (Ω,F)to the state space (E ,G). It is a function of two variables, t ∈ Tand ω ∈ Ω.

For a fixed ω ∈ Ω the function Xt(ω), t ∈ T is the sample path ofthe process X associated with ω.

Let K be a collection of subsets of Ω. The smallest σ–algebra onΩ which contains K is denoted by σ(K) and is called theσ–algebra generated by K.

Let Xt : Ω 7→ E , t ∈ T . The smallest σ–algebra σ(Xt , t ∈ T ), suchthat the family of mappings Xt , t ∈ T is a stochastic processwith sample space (Ω, σ(Xt , t ∈ T )) and state space (E ,G), iscalled the σ–algebra generated by Xt , t ∈ T.


A filtration on (Ω,F) is a nondecreasing family Ft , t ∈ T ofsub–σ–algebras of F : Fs ⊆ Ft ⊆ F for s 6 t .We set F∞ = σ(∪t∈TFt). The filtration generated by Xt , whereXt is a stochastic process, is

FXt := σ (Xs; s 6 t) .

We say that a stochastic process Xt ; t ∈ T is adapted to thefiltration Ft := Ft , t ∈ T if for all t ∈ T , Xt is an Ft–measurablerandom variable.

DefinitionLet Xt be a stochastic process defined on a probability space(Ω,F , µ) with values in E and let FX

t be the filtration generated byXt. Then Xt is a Markov process if

P(Xt ∈ Γ|FXs ) = P(Xt ∈ Γ|Xs) (16)

for all t , s ∈ T with t > s, and Γ ∈ B(E).


The filtration FXt is generated by events of the form

ω|Xs1 ∈ B1, Xs2 ∈ B2, . . .Xsn ∈ Bn, with0 6 s1 < s2 < · · · < sn 6 s and Bi ∈ B(E). The definition of aMarkov process is thus equivalent to the hierarchy of equations

P(Xt ∈ Γ|Xt1 ,Xt2 , . . .Xtn) = P(Xt ∈ Γ|Xtn) a.s.

for n > 1, 0 6 t1 < t2 < · · · < tn 6 t and Γ ∈ B(E).


Roughly speaking, the statistics of Xt for t > s are completelydetermined once Xs is known; information about Xt for t < s issuperfluous. In other words: a Markov process has no memory .More precisely: when a Markov process is conditioned on thepresent state, then there is no memory of the past. The past andfuture of a Markov process are statistically independent when thepresent is known.

A typical example of a Markov process is the random walk: inorder to find the position x(t + 1) of the random walker at timet + 1 it is enough to know its position x(t) at time t : how it got tox(t) is irrelevant.

A non Markovian process Xt can be described through aMarkovian one Yt by enlarging the state space: the additionalvariables that we introduce account for the memory in the Xt . This"Markovianization" trick is very useful since there are many moretools for analyzing Markovian process.


With a Markov process Xt we can associate a functionP : T × T × E × B(E) → R

+ defined through the relation

P

[Xt ∈ Γ|FX

s

]= P(s, t ,Xs ,Γ),

for all t , s ∈ T with t > s and all Γ ∈ B(E).

Assume that Xs = x . Since P[Xt ∈ Γ|FX

s

]= P [Xt ∈ Γ|Xs] we can

writeP(Γ, t |x , s) = P [Xt ∈ Γ|Xs = x ] .

The transition function P(t ,Γ|x , s) is (for fixed t , x s) aprobability measure on E with P(t ,E |x , s) = 1; it isB(E)–measurable in x (for fixed t , s, Γ) and satisfies theChapman–Kolmogorov equation

P(Γ, t |x , s) =

∫

EP(Γ, t |y ,u)P(dy ,u|x , s). (17)

for all x ∈ E , Γ ∈ B(E) and s,u, t ∈ T with s 6 u 6 t .


The derivation of the Chapman-Kolmogorov equation is based onthe assumption of Markovianity and on properties of theconditional probability:

1 Let (Ω,F , µ) be a probability space, X a random variable from(Ω,F , µ) to (E ,G) and let F1 ⊂ F2 ⊂ F . Then

E(E(X |F2)|F1) = E(E(X |F1)|F2) = E(X |F1). (18)

2 Given G ⊂ F we define the function PX (B|G) = P(X ∈ B|G) forB ∈ F . Assume that f is such that E(f (X)) <∞. Then

E(f (X)|G) =

∫

R

f (x)PX (dx |G). (19)


The CK equation is an integral equation and is the fundamentalequation in the theory of Markov processes. Under additionalassumptions we will derive from it the Fokker-Planck PDE, whichis the fundamental equation in the theory of diffusion processes,and will be the main object of study in this course.

A Markov process is homogeneous if

P(t ,Γ|Xs = x) := P(s, t , x ,Γ) = P(0, t − s, x ,Γ).

We set P(0, t , ·, ·) = P(t , ·, ·). The Chapman–Kolmogorov (CK)equation becomes

P(t + s, x ,Γ) =

∫

EP(s, x ,dz)P(t , z,Γ). (20)


Let Xt be a homogeneous Markov process and assume that theinitial distribution of Xt is given by the probability measureν(Γ) = P(X0 ∈ Γ) (for deterministic initial conditions–X0 = x– wehave that ν(Γ) = IΓ(x) ).The transition function P(x , t ,Γ) and the initial distribution νdetermine the finite dimensional distributions of X by

P(X0 ∈ Γ1, X (t1) ∈ Γ1, . . . ,Xtn ∈ Γn)

=

∫

Γ0

∫

Γ1

. . .

∫

Γn−1

P(tn − tn−1, yn−1,Γn)P(tn−1 − tn−2, yn−2,dyn−1)

· · · × P(t1, y0,dy1)ν(dy0). (21)

Theorem(Ethier and Kurtz 1986, Sec. 4.1) Let P(t , x ,Γ) satisfy (20) andassume that (E , ρ) is a complete separable metric space. Then thereexists a Markov process X in E whose finite-dimensional distributionsare uniquely determined by (21).


Let Xt be a homogeneous Markov process with initial distributionν(Γ) = P(X0 ∈ Γ) and transition function P(x , t ,Γ). We can calculatethe probability of finding Xt in a set Γ at time t :

P(Xt ∈ Γ) =

∫

EP(x , t ,Γ)ν(dx).

Thus, the initial distribution and the transition function are sufficient tocharacterize a homogeneous Markov process. Notice that they do notprovide us with any information about the actual paths of the Markovprocess.


The transition probability P(Γ, t |x , s) is a probability measure.Assume that it has a density for all t > s:

P(Γ, t |x , s) =

∫

Γp(y , t |x , s) dy .

Clearly, for t = s we have P(Γ, s|x , s) = IΓ(x).

The Chapman-Kolmogorov equation becomes:∫

Γp(y , t |x , s) dy =

∫

R

∫

Γp(y , t |z,u)p(z,u|x , s) dzdy ,

and, since Γ ∈ B(R) is arbitrary, we obtain the equation

p(y , t |x , s) =

∫

R

p(y , t |z,u)p(z,u|x , s) dz. (22)

The transition probability density is a function of 4 arguments: theinitial position and time x , s and the final position and time y , t .


In words, the CK equation tells us that, for a Markov process, thetransition from x , s to y , t can be done in two steps: first the systemmoves from x to z at some intermediate time u. Then it moves from zto y at time t . In order to calculate the probability for the transition from(x , s) to (y , t) we need to sum (integrate) the transitions from allpossible intermediary states z. The above description suggests that aMarkov process can be described through a semigroup of operators,i.e. a one-parameter family of linear operators with the properties

P0 = I, Pt+s = Pt Ps ∀ t , s > 0.

A semigroup of operators is characterized through its generator .


Indeed, let P(t , x ,dy) be the transition function of a homogeneousMarkov process. It satisfies the CK equation (20):

P(t + s, x ,Γ) =

∫

EP(s, x ,dz)P(t , z,Γ).

Let X := Cb(E) and define the operator

(Pt f )(x) := E(f (Xt)|X0 = x) =

∫

Ef (y)P(t , x ,dy).

This is a linear operator with

(P0f )(x) = E(f (X0)|X0 = x) = f (x) ⇒ P0 = I.


Furthermore:

(Pt+sf )(x) =

∫f (y)P(t + s, x ,dy)

=

∫ ∫f (y)P(s, z,dy)P(t , x ,dz)

=

∫ (∫f (y)P(s, z,dy)

)P(t , x ,dz)

=

∫(Psf )(z)P(t , x ,dz)

= (Pt Psf )(x).

Consequently:Pt+s = Pt Ps.


Let (E , ρ) be a metric space and let Xt be an E -valuedhomogeneous Markov process. Define the one parameter familyof operators Pt through

Pt f (x) =

∫f (y)P(t , x ,dy) = E[f (Xt)|X0 = x ]

for all f (x) ∈ Cb(E) (continuous bounded functions on E ).

Assume for simplicity that Pt : Cb(E) → Cb(E). Then theone-parameter family of operators Pt forms a semigroup ofoperators on Cb(E).

We define by D(L) the set of all f ∈ Cb(E) such that the stronglimit

Lf = limt→0

Pt f − ft

,

exists.


DefinitionThe operator L : D(L) → Cb(E) is called the infinitesimal generatorof the operator semigroup Pt .

DefinitionThe operator L : Cb(E) → Cb(E) defined above is called the generatorof the Markov process Xt.

The study of operator semigroups started in the late 40’sindependently by Hille and Yosida. Semigroup theory wasdeveloped in the 50’s and 60’s by Feller, Dynkin and others,mostly in connection to the theory of Markov processes.

Necessary and sufficient conditions for an operator L to be thegenerator of a (contraction) semigroup are given by theHille-Yosida theorem (e.g. Evans Partial Differential Equations,AMS 1998, Ch. 7).


The semigroup property and the definition of the generator of asemigroup imply that, formally at least, we can write:

Pt = exp(Lt).

Consider the function u(x , t) := (Pt f )(x). We calculate its timederivative:

∂u∂t

=ddt

(Pt f ) =ddt

(eLt f

)

= L(eLt f

)= LPt f = Lu.

Furthermore, u(x ,0) = P0f (x) = f (x). Consequently, u(x , t)satisfies the initial value problem

∂u∂t

= Lu, u(x ,0) = f (x). (23)


When the semigroup Pt is the transition semigroup of a Markovprocess Xt , then equation (23) is called the backwardKolmogorov equation . It governs the evolution of an observable

u(x , t) = E(f (Xt)|X0 = x).

Thus, given the generator of a Markov process L, we cancalculate all the statistics of our process by solving the backwardKolmogorov equation.

In the case where the Markov process is the solution of astochastic differential equation, then the generator is a secondorder elliptic operator and the backward Kolmogorov equationbecomes an initial value problem for a parabolic PDE.


The space Cb(E) is natural in a probabilistic context, but otherBanach spaces often arise in applications; in particular whenthere is a measure µ on E , the spaces Lp(E ;µ) sometimes arise.We will quite often use the space L2(E ;µ), where µ will is theinvariant measure of our Markov process.

The generator is frequently taken as the starting point for thedefinition of a homogeneous Markov process.

Conversely, let Pt be a contraction semigroup (Let X be aBanach space and T : X → X a bounded operator. Then T is acontraction provided that ‖Tf‖X 6 ‖f‖X ∀ f ∈ X ), withD(Pt) ⊂ Cb(E), closed. Then, under mild technical hypotheses,there is an E–valued homogeneous Markov process Xtassociated with Pt defined through

E[f (X (t)|FXs )] = Pt−sf (X (s))

for all t , s ∈ T with t > s and f ∈ D(Pt).


ExampleThe one dimensional Brownian motion is a homogeneous Markovprocess. The transition function is the Gaussian defined in theexample in Lecture 2:

P(t , x ,dy) = γt,x(y)dy , γt,x (y) =1√2πt

exp(−|x − y |2

2t

).

The semigroup associated to the standard Brownian motion is the heat

semigroup Pt = et2

d2

dx2 . The generator of this Markov process is 12

d2

dx2 .

Notice that the transition probability density γt,x of the onedimensional Brownian motion is the fundamental solution (Green’sfunction) of the heat (diffusion) PDE

∂γ

∂t=

12∂2γ

∂x2 .


Example

The Ornstein-Uhlenbeck process Vt = e−tW (e2t) is a homogeneousMarkov process. The transition probability density is the Gaussian

p(y , t |x , s) := p(Vt = y |Vs = x)

=1√

2π(1 − e−2(t−s))exp

(− |y − xe−(t−s)|2

2(1 − e−2(t−s))

).

The semigroup associated to the Ornstein-Uhlenbeck process is

Pt = et(−x ddx + 1

2d2

dx2 ). The generator of this Markov process isL = −x d

dx + 12

d2

dx2 .

Notice that the transition probability density p(t , x) of the 1dOU process is the fundamental solution (Green’s function) of theheat (diffusion) PDE

∂p∂t

=∂(xp)

∂x+

12∂2p∂x2 .


The semigroup Pt acts on bounded measurable functions.

We can also define the adjoint semigroup P∗t which acts on

probability measures:

P∗t µ(Γ) =

∫

R

P(Xt ∈ Γ|X0 = x) dµ(x) =

∫

R

p(t , x ,Γ) dµ(x).

The image of a probability measure µ under P∗t is again a

probability measure. The operators Pt and P∗t are adjoint in the

L2-sense: ∫

R

Pt f (x) dµ(x) =

∫

R

f (x) d(P∗t µ)(x). (24)


We can, formally at least, write

P∗t = exp(L∗t),

where L∗ is the L2-adjoint of the generator of the process:∫

Lfh dx =

∫fL∗h dx .

Let µt := P∗t µ. This is the law of the Markov process and µ is the

initial distribution. An argument similar to the one used in thederivation of the backward Kolmogorov equation (23) enables usto obtain an equation for the evolution of µt :

∂µt

∂t= L∗µt , µ0 = µ.


Assuming that µt = ρ(y , t) dy , µ = ρ0(y) dy this equationbecomes:

∂ρ

∂t= L∗ρ, ρ(y ,0) = ρ0(y). (25)

This is the forward Kolmogorov or Fokker-Planck equation.When the initial conditions are deterministic, X0 = x , the initialcondition becomes ρ0 = δ(y − x).

Given the initial distribution and the generator of the Markovprocess Xt , we can calculate the transition probability density bysolving the Forward Kolmogorov equation. We can then calculateall statistical quantities of this process through the formula

E(f (Xt)|X0 = x) =

∫f (y)ρ(t , y ; x) dy .

We will derive rigorously the backward and forward Kolmogorovequations for Markov processes that are defined as solutions ofstochastic differential equations later on.


We can study the evolution of a Markov process in two differentways:

Either through the evolution of observables(Heisenberg/Koopman)

∂(Pt f )∂t

= L(Pt f ),

or through the evolution of states(Schrödinger/Frobenious-Perron)

∂(P∗t µ)

∂t= L∗(P∗

t µ).

We can also study Markov processes at the level of trajectories.We will do this after we define the concept of a stochasticdifferential equation.


A very important concept in the study of limit theorems forstochastic processes is that of ergodicity .

This concept, in the context of Markov processes, provides us withinformation on the long–time behavior of a Markov semigroup.

DefinitionA Markov process is called ergodic if the equation

Ptg = g, g ∈ Cb(E) ∀t > 0

has only constant solutions.

Roughly speaking, ergodicity corresponds to the case where thesemigroup Pt is such that Pt − I has only constants in its nullspace, or, equivalently, to the case where the generator L hasonly constants in its null space. This follows from the definition ofthe generator of a Markov process.


Under some additional compactness assumptions, an ergodicMarkov process has an invariant measure µ with the property that,in the case T = R

+,

limt→+∞

1t

∫ t

0g(Xs) ds = Eg(x),

where E denotes the expectation with respect to µ.

This is a physicist’s definition of an ergodic process: timeaverages equal phase space averages .

Using the adjoint semigroup we can define an invariant measureas the solution of the equation

P∗t µ = µ.

If this measure is unique, then the Markov process is ergodic.


Using this, we can obtain an equation for the invariant measure interms of the adjoint of the generator L∗, which is the generator ofthe semigroup P∗

t . Indeed, from the definition of the generator of asemigroup and the definition of an invariant measure, we concludethat a measure µ is invariant if and only if

L∗µ = 0

in some appropriate generalized sense ((L∗µ, f ) = 0 for everybounded measurable function).

Assume that µ(dx) = ρ(x) dx . Then the invariant densitysatisfies the stationary Fokker-Planck equation

L∗ρ = 0.

The invariant measure (distribution) governs the long-timedynamics of the Markov process.


If X0 is distributed according to µ, then so is Xt for all t > 0. Theresulting stochastic process, with X0 distributed in this way, isstationary .

In this case the transition probability density (the solution of theFokker-Planck equation) is independent of time: ρ(x , t) = ρ(x).

Consequently, the statistics of the Markov process is independentof time.


ExampleThe one dimensional Brownian motion is not an ergodic process: Thenull space of the generator L = 1

2d2

dx2 on R is not one dimensional!

ExampleConsider a one-dimensional Brownian motion on [0,1], with periodicboundary conditions. The generator of this Markov process L is thedifferential operator L = 1

2d2

dx2 , equipped with periodic boundaryconditions on [0,1]. This operator is self-adjoint. The null space of bothL and L∗ comprises constant functions on [0,1]. Both the backwardKolmogorov and the Fokker-Planck equation reduce to the heatequation

∂ρ

∂t=

12∂2ρ

∂x2

with periodic boundary conditions in [0,1]. Fourier analysis shows thatthe solution converges to a constant at an exponential rate.


ExampleThe one dimensional Ornstein-Uhlenbeck (OU) process is aMarkov process with generator

L = −αxddx

+ Dd2

dx2 .

The null space of L comprises constants in x . Hence, it is anergodic Markov process. In order to calculate the invariantmeasure we need to solve the stationary Fokker–Planck equation:

L∗ρ = 0, ρ > 0, ‖ρ‖L1(R) = 1. (26)


Example (Continued)

Let us calculate the L2-adjoint of L. Assuming that f , h decaysufficiently fast at infinity, we have:

∫

R

Lfh dx =

∫

R

[(−αx∂x f )h + (D∂2

x f )h]

dx

=

∫

R

[f∂x(αxh) + f (D∂2

x h)]

dx =:

∫

R

fL∗h dx ,

where

L∗h :=ddx

(axh) + Dd2hdx2 .

We can calculate the invariant distribution by solvingequation (26).

The invariant measure of this process is the Gaussian measure

µ(dx) =

√α

2πDexp

(− α

2Dx2)

dx .

If the initial condition of the OU process is distributed according toPavliotis (IC) StochProc January 16, 2011 112 / 367

Let Xt be the 1d OU process and let X0 ∼ N (0,D/α). Then Xt is amean zero, Gaussian second order stationary process on [0,∞)with correlation function

R(t) =Dα

e−α|t|

and spectral density

f (x) =Dπ

1x2 + α2 .

Furthermore, the OU process is the only real-valued mean zeroGaussian second-order stationary Markov process defined on R.


DIFFUSION PROCESSES


A Markov process consists of three parts: a drift (deterministic), arandom process and a jump process.

A diffusion process is a Markov process that has continuoussample paths (trajectories). Thus, it is a Markov process with nojumps.

A diffusion process can be defined by specifying its first twomoments:


Definition

A Markov process Xt with transition probability P(Γ, t |x , s) is called adiffusion process if the following conditions are satisfied.

1 (Continuity). For every x and every ε > 0∫

|x−y |>εP(dy , t |x , s) = o(t − s) (27)

uniformly over s < t .2 (Definition of drift coefficient). There exists a function a(x , s) such

that for every x and every ε > 0∫

|y−x|6ε(y − x)P(dy , t |x , s) = a(x , s)(s − t) + o(s − t). (28)

uniformly over s < t .


Definition (Continued)3. (Definition of diffusion coefficient). There exists a function b(x , s)

such that for every x and every ε > 0∫

|y−x|6ε(y − x)2P(dy , t |x , s) = b(x , s)(s − t) + o(s − t). (29)

uniformly over s < t .


In Definition 39 we had to truncate the domain of integration since wedidn’t know whether the first and second moments exist. If we assumethat there exists a δ > 0 such that

limt→s

1t − s

∫

Rd|y − x |2+δP(s, x , t ,dy) = 0, (30)

then we can extend the integration over the whole Rd and use

expectations in the definition of the drift and the diffusion coefficient.Indeed, ,let k = 0,1,2 and notice that

∫

|y−x|>ε|y − x |kP(s, x , t ,dy)

=

∫

|y−x|>ε|y − x |2+δ|y − x |k−(2+δ)P(s, x , t ,dy)

61

ε2+δ−k

∫

|y−x|>ε|y − x |2+δP(s, x , t ,dy)

61

ε2+δ−k

∫

Rd|y − x |2+δP(s, x , t ,dy).


Using this estimate together with (30) we conclude that:

limt→s

1t − s

∫

|y−x|>ε|y − x |k P(s, x , t ,dy) = 0, k = 0,1,2.

This implies that assumption (30) is sufficient for the sample paths tobe continuous (k = 0) and for the replacement of the truncatedintegrals in (28) and (29) by integrals over R

d (k = 1 and k = 2,respectively). The definitions of the drift and diffusion coefficientsbecome:

limt→s

E

(Xt − Xs

t − s

∣∣∣Xs = x)

= a(x , s) (31)

and

limt→s

E

((Xt − Xs) ⊗ (Xt − Xs)

t − s

∣∣∣Xs = x)

= b(x , s) (32)


Notice also that the continuity condition can be written in the form

P (|Xt − Xs| > ε|Xs = x) = o(t − s).

Now it becomes clear that this condition implies that the probability oflarge changes in Xt over short time intervals is small. Notice, on theother hand, that the above condition implies that the sample paths of adiffusion process are not differentiable : if they where, then the righthand side of the above equation would have to be 0 when t − s ≪ 1.The sample paths of a diffusion process have the regularity ofBrownian paths. A Markovian process cannot be differentiable: wecan define the derivative of a sample paths only with processes forwhich the past and future are not statistically independent whenconditioned on the present.


Let us denote the expectation conditioned on Xs = x by Es,x . Notice

that the definitions of the drift and diffusion coefficients (31) and (32)can be written in the form

Es,x(Xt − Xs) = a(x , s)(t − s) + o(t − s).

and

Es,x((Xt − Xs) ⊗ (Xt − Xs)

)= b(x , s)(t − s) + o(t − s).

Consequently, the drift coefficient defines the mean velocity vectorfor the stochastic process Xt , whereas the diffusion coefficient (tensor)is a measure of the local magnitude of fluctuations of Xt − Xs about themean value. hence, we can write locally:

Xt − Xs ≈ a(s,Xs)(t − s) + σ(s,Xs) ξt ,

where b = σσT and ξt is a mean zero Gaussian process with

Es,x(ξt ⊗ ξs) = (t − s)I.


Since we have that

Wt − Ws ∼ N (0, (t − s)I),

we conclude that we can write locally:

∆Xt ≈ a(s,Xs)∆t + σ(s,Xs)∆Wt .

Or, replacing the differences by differentials:

dXt = a(t ,Xt)dt + σ(t ,Xt )dWt .

Hence, the sample paths of a diffusion process are governed by astochastic differential equation (SDE).


Theorem

(Kolmogorov) Let f (x) ∈ Cb(R) and let

u(x , s) := E(f (Xt)|Xs = x) =

∫f (y)P(dy , t |x , s) ∈ C2

b(R).

Assume furthermore that the functions a(x , s), b(x , s) are continuousin both x and s. Then u(x , s) ∈ C2,1(R × R

+) and it solves the finalvalue problem

− ∂u∂s

= a(x , s)∂u∂x

+12

b(x , s)∂2u∂x2 , lim

s→tu(s, x) = f (x). (33)


Proof:First we notice that, the continuity assumption (27), together with thefact that the function f (x) is bounded imply that

u(x , s) =

∫

R

f (y) P(dy , t |x , s)

=

∫

|y−x|6εf (y)P(dy , t |x , s) +

∫

|y−x|>εf (y)P(dy , t |x , s)

6

∫

|y−x|6εf (y)P(dy , t |x , s) + ‖f‖L∞

∫

|y−x|>εP(dy , t |x , s)

=

∫

|y−x|6εf (y)P(dy , t |x , s) + o(t − s).


Now we show that u(s, x) solves the backward Kolmogorov equation.We use the Chapman-Kolmogorov equation (17) to obtain

u(x , σ) =

∫

R

f (z)P(dz, t |x , σ) (34)

=

∫

R

∫

R

f (z)P(dz, t |y , ρ)P(dy , ρ|x , σ)

=

∫

R

u(y , ρ)P(dy , ρ|x , σ). (35)

The Taylor series expansion of the function u(s, x) gives

u(z, ρ)−u(x , ρ) =∂u(x , ρ)∂x

(z−x)+12∂2u(x , ρ)∂x2 (z−x)2(1+αε), |z−x | 6 ε,

(36)where

αε = supρ,|z−x|6ε

∣∣∣∣∂2u(x , ρ)∂x2 − ∂2u(z, ρ)

∂x2

∣∣∣∣ .

Notice that, since u(x , s) is twice continuously differentiable in x ,limε→0 αε = 0.


We combine now (35) with (36) to calculate

u(x , s) − u(x , s + h)

h=

1h

(∫

R

P(dy , s + h|x , s)u(y , s + h) − u(x , s + h)

)

=1h

∫

R

P(dy , s + h|x , s)(u(y , s + h) − u(x , s + h))

=1h

∫

|x−y |<εP(dy , s + h|x , s)(u(y , s + h) − u(x , s)) + o(1)

=∂u∂x

(x , s + h)1h

∫

|x−y |<ε(y − x)P(dy , s + h|x , s)

+12∂2u∂x2 (x , s + h)

1h

∫

|x−y |<ε(y − x)2P(dy , s + h|x , s)(1 + αε) + o(1)

= a(x , s)∂u∂x

(x , s + h) +12

b(x , s)∂2u∂x2 (x , s + h)(1 + αε) + o(1).

Equation (33) follows by taking the limits ε→ 0, h → 0.Pavliotis (IC) StochProc January 16, 2011 127 / 367

Assume now that the transition function has a density p(y , t |x , s). Inthis case the formula for u(x , s) becomes

u(x , s) =

∫

R

f (y)p(y , t |x , s) dy .

Substituting this in the backward Kolmogorov equation we obtain∫

R

f (y)

(∂p(y , t |x , s)

∂s+ As,xp(y , t |x , s)

)= 0 (37)

where

As,x := a(x , s)∂

∂x+

12

b(x , s)∂2

∂x2 .

Since (37) is valid for arbitrary functions f (y), we obtain a partialdifferential equations for the transition probability density:

− ∂p(y , t |x , s)

∂s= a(x , s)

∂p(y , t |x , s)

∂x+

12

b(x , s)∂2p(y , t |x , s)

∂x2 . (38)

Notice that the variation is with respect to the "backward" variablesx , s. We can also obtain an equation with respect to the "forward"variables y , t , the Forward Kolmogorov equation .


Now we derive the forward Kolmogorov equation or Fokker-Planckequation . We assume that the transition function has a density withrespect to Lebesgue measure.

P(Γ, t |x , s) =

∫

Γp(t , y |x , s) dy .

Theorem

(Kolmogorov) Assume that conditions (27), (28), (29) are satisfied andthat p(y , t |·, ·), a(y , t),b(y , t) ∈ C2,1(R × R

+). Then the transitionprobability density satisfies the equation

∂p∂t

= − ∂

∂y(a(t , y)p) +

12∂2

∂y2 (b(t , y)p) , limt→s

p(t , y |x , s) = δ(x − y).

(39)


ProofFix a function f (y) ∈ C2

0(R). An argument similar to the one used in theproof of the backward Kolmogorov equation gives

limh→0

1h

(∫f (y)p(y , s + h|x , s) ds − f (x)

)= a(x , s)fx (x)+

12

b(x , s)fxx (x),

(40)where subscripts denote differentiation with respect to x . On the otherhand


In the above calculation used the Chapman-Kolmogorov equation. Wehave also performed two integrations by parts and used the fact that,since the test function f has compact support, the boundary termsvanish.Since the above equation is valid for every test function f (y), theforward Kolmogorov equation follows.


Assume now that initial distribution of Xt is ρ0(x) and set s = 0 (theinitial time) in (39). Define

p(y , t) :=

∫p(y , t |x ,0)ρ0(x) dx .

We multiply the forward Kolmogorov equation (39) by ρ0(x) andintegrate with respect to x to obtain the equation

∂p(y , t)∂t

= − ∂

∂y(a(y , t)p(y , t)) +

12∂2

∂y2 (b(y , t)p(t , y)) , (41)

together with the initial condition

p(y ,0) = ρ0(y). (42)

The solution of equation (41), provides us with the probability that thediffusion process Xt , which initially was distributed according to theprobability density ρ0(x), is equal to y at time t . Alternatively, we canthink of the solution to (39) as the Green’s function for the PDE (41).


Quite often we need to calculate joint probability densities. For,example the probability that Xt1 = x1 and Xt2 = x2. From the propertiesof conditional expectation we have that

p(x1, t1, x2, t2) = PXt1 = x1,Xt2 = x2)

= P(Xt1 = x1|Xt2 = x2)P(Xt2 = x2)

= p(x1, t1|x2t2)p(x2, t2).

Using the joint probability density we can calculate the statistics of afunction of the diffusion process Xt at times t and s:

E(f (Xt ,Xs)) =

∫ ∫f (y , x)p(y , t |x , s)p(x , s) dxdy . (43)

The autocorrelation function at time t and s is given by

E(XtXs) =

∫ ∫yxp(y , t |x , s)p(x , s) dxdy .

In particular,

E(XtX0) =

∫ ∫yxp(y , t |x ,0)p(x ,0) dxdy .


The drift and diffusion coefficients of a diffusion process in Rd are

defined as:

limt→s

1t − s

∫

|y−x|<ε(y − x)P(dy , t |x , s) = a(x , s)

and

limt→s

1t − s

∫

|y−x|<ε(y − x) ⊗ (y − x)P(dy , t |x , s) = b(x , s).

The drift coefficient a(x , s) is a d -dimensional vector field and thediffusion coefficient b(x , s) is a d × d symmetric matrix (second ordertensor). The generator of a d dimensional diffusion process is

L = a(s, x) · ∇ +12

b(s, x) : ∇∇

=

d∑

j=1

aj∂

∂xj+

12

d∑

i ,j=1

bij∂2

∂x2j

.


Assuming that the first and second moments of the multidimensionaldiffusion process exist, we can write the formulas for the drift vectorand diffusion matrix as

limt→s

E

(Xt − Xs

t − s

∣∣∣Xs = x)

= a(x , s) (44)

and

limt→s

E

((Xt − Xs) ⊗ (Xt − Xs)

t − s

∣∣∣Xs = x)

= b(x , s) (45)

Notice that from the above definition it follows that the diffusion matrixis symmetric and nonnegative definite.


THE FOKKER-PLANCK EQUATION


Consider a diffusion process on Rd with time-independent drift

and diffusion coefficients. The Fokker-Planck equation is

∂p∂t

= −d∑

j=1

∂

∂xj(ai(x)p) +

12

d∑

i ,j=1

∂2

∂xi∂xj(bij(x)p), t > 0, x ∈ R

d ,

(46a)p(x ,0) = f (x), x ∈ R

d . (46b)

Write it in non-divergence form :

∂p∂t

=d∑

j=1

aj(x)∂p∂xj

+12

d∑

i ,j=1

bij(x)∂2p∂xi∂xj

+ c(x)u, t > 0, x ∈ Rd ,

(47a)p(x ,0) = f (x), x ∈ R

d , (47b)


where

ai(x) = −ai(x) +

d∑

j=1

∂bij

∂xj, ci(x) =

12

d∑

i ,j=1

∂2bij

∂xi∂xj−

d∑

i=1

∂ai

∂xi.

The diffusion matrix is always nonnegative. We will assume that itis actually positive, i.e. we will impose the uniform ellipticitycondition:

d∑

i ,j=1

bij(x)ξiξj > α‖ξ‖2, ∀ ξ ∈ Rd , (48)

Furthermore, we will assume that the coefficients a, b, c aresmooth and that they satisfy the growth conditions

‖b(x)‖ 6 M, ‖a(x)‖ 6 M(1 + ‖x‖), ‖c(x)‖ 6 M(1 + ‖x‖2). (49)


We will call a solution to the Cauchy problem for theFokker–Planck equation (47) a classical solution if:

1 u ∈ C2,1(Rd ,R+).2 ∀T > 0 there exists a c > 0 such that

‖u(t , x)‖L∞(0,T ) 6 ceα‖x‖2

3 limt→0 u(t , x) = f (x).


TheoremAssume that conditions (48) and (49) are satisfied, and assume that|f | 6 ceα‖x‖2

. Then there exists a unique classical solution to theCauchy problem for the Fokker–Planck equation. Furthermore, thereexist positive constants K , δ so that

|p|, |pt |, ‖∇p‖, ‖D2p‖ 6 Kt(−n+2)/2 exp(− 1

2tδ‖x‖2

).

This estimate enables us to multiply the Fokker-Planck equationby monomials xn and then to integrate over R

d and to integrate byparts.

For a proof of this theorem see Friedman, Partial DifferentialEquations of Parabolic Type, Prentice-Hall 1964.


The FP equation as a conservation law

We can define the probability current to be the vector whose i thcomponent is

Ji := ai(x)p − 12

d∑

j=1

∂

∂xj

(bij(x)p

).

The Fokker–Planck equation can be written as a continuityequation :

∂p∂t

+ ∇ · J = 0.

Integrating the FP equation over Rd and integrating by parts on

the right hand side of the equation we obtain

ddt

∫

Rdp(x , t) dx = 0.

Consequently:

‖p(·, t)‖L1(Rd ) = ‖p(·,0)‖L1(Rd ) = 1.


Boundary conditions for the Fokker–Planckequation

So far we have been studying the FP equation on Rd .

The boundary condition was that the solution decays sufficientlyfast at infinity.

For ergodic diffusion processes this is equivalent to requiring thatthe solution of the backward Kolmogorov equation is an elementof L2(µ) where µ is the invariant measure of the process.

We can also study the FP equation in a bounded domain withappropriate boundary conditions.

We can have absorbing, reflecting or periodic boundaryconditions.


Consider the FP equation posed in Ω ⊂ Rd where Ω is a bounded

domain with smooth boundary.Let J denote the probability current and let n be the unit normalvector to the surface.

1 We specify reflecting boundary conditions by setting

n · J(x , t) = 0, on ∂Ω.

2 We specify absorbing boundary conditions by setting

p(x , t) = 0, on ∂Ω.

3 When the coefficient of the FP equation are periodic functions, wemight also want to consider periodic boundary conditions, i.e. thesolution of the FP equation is periodic in x with period equal to thatof the coefficients.


Reflecting BC correspond to the case where a particle whichevolves according to the SDE corresponding to the FP equationgets reflected at the boundary.

Absorbing BC correspond to the case where a particle whichevolves according to the SDE corresponding to the FP equationgets absorbed at the boundary.

There is a complete classification of boundary conditions in onedimension, the Feller classification : the BC can be regular, exit,entrance and natural .


Examples of Diffusion Processes

Set a(t , x) ≡ 0, b(t , x) ≡ 2D > 0. The Fokker-Planck equationbecomes:

∂p∂t

= D∂2p∂x2 , p(x , s|y , s) = δ(x − y).

This is the heat equation, which is the Fokker-Planck equation forBrownian motion (Einstein, 1905). Its solution is

pW (x , t |y , s) =1√

2πD(t − s)exp

(− (x − s)2

2D(t − s)

).


Assume that the initial distribution is

pW (x , s|y , s) = W (y , s).

The solution of the Fokker-Planck equation for Brownian motionwith this initial distribution is

PW (x , t) =

∫p(x , t |y , s)W (y , s) dy .

The Gaussian distribution is the fundamental solution (Green’sfunction) of the heat equation (i.e. the Fokker-Planck equation forBrownian motion).


Set a(t , x) = −αx , b(t , x) ≡ 2D > 0:

∂p∂t

= α∂(xp)

∂x+ D

∂2p∂x2 .

This is the Fokker-Planck equation for the Ornstein-Uhlenbeckprocess (Ornstein-Uhlenbeck, 1930). Its solution is

pOU(x , t |y , s) =

√α

2πD(1 − e−2α(t−s))exp

(−α(x − e−α(t−s)y)2

2D(1 − e−2α(t−s))

).

Proof: take the Fourier transform in x , use the method ofcharacteristics and take the inverse Fourier transform.


Set y = 0, s = 0. Notice that

limα→0

pOU(x , t) = pW (x , t).

Thus, in the limit where the friction coefficient goes to 0, werecover distribution function of BM from the DF of the OUprocesses.

Notice also that

limt→+∞

pOU(x , t) =

√α

2πDexp

(−αx2

2D

).

Thus, the Ornstein-Uhlenbeck process is an ergodic Markovprocess. Its invariant measure is Gaussian.

We can calculate all moments of the OU process.


Define the nth moment of the OU process:

Mn =

∫

R

xnp(x , t) dx , n = 0,1,2, . . . .

Let n = 0. We integrate the FP equation over R to obtain:∫∂p∂t

= α

∫∂(xp)

∂x+ D

∫∂2p∂x2 = 0,

after an integration by parts and using the fact that p(x , t) decayssufficiently fast at infinity. Consequently:

ddt

M0 = 0 ⇒ M0(t) = M0(0) = 1.

In other words:

ddt

‖p‖L1(R) = 0 ⇒ p(x , t) = p(x , t = 0).

Consequently: probability is conserved .


Let n = 1. We multiply the FP equation for the OU process by x ,integrate over R and perform and integration by parts to obtain:

ddt

M1 = −αM1.

Consequently, the first moment converges exponentially fast to0:

M1(t) = e−αtM1(0).


Let now n > 2. We multiply the FP equation for the OU process byxn and integrate by parts (once on the first term on the RHS andtwice on the second) to obtain:

ddt

∫xnp = −αn

∫xnp + Dn(n − 1)

∫xn−2p.

Or, equivalently:

ddt

Mn = −αnMn + Dn(n − 1)Mn−2, n > 2.

This is a first order linear inhomogeneous differential equation.We can solve it using the variation of constants formula:

Mn(t) = e−αntMn(0) + Dn(n − 1)

∫ t

0e−αn(t−s)Mn−2(s) ds.


The stationary moments of the OU process are:

〈xn〉OU =

√α

2πD

∫

R

xne−αx2

2D dx

=

1.3 . . . (n − 1)(Dα

)n/2, n even,

0, n odd.

We have thatlim

t→∞Mn(t) = 〈xn〉OU .

If the initial conditions of the OU process are stationary, then:

Mn(t) = Mn(0) = 〈xn〉OU .


set a(x) = µx , b(x) = 12σx2. This is the geometric Brownian

motion . The generator of this process is

L = µx∂

∂x+σ2x2

2∂2

∂x2 .

Notice that this operator is not uniformly elliptic.

The Fokker-Planck equation of the geometric Brownian motion is:

∂p∂t

= − ∂

∂x(µx) +

∂2

∂x2

(σ2x2

2p).


We can easily obtain an equation for the nth moment of thegeometric Brownian motion:

ddt

Mn =(µn +

σ2

2n(n − 1)

)Mn, n > 2.

The solution of this equation is

Mn(t) = e(µ+(n−1)σ2

2 )ntMn(0), n > 2

andM1(t) = eµtM1(0).

Notice that the nth moment might diverge as t → ∞, dependingon the values of µ and σ:

limt→∞

Mn(t) =

0, σ2

2 < − µn−1 ,

Mn(0), σ2

2 = − µn−1 ,

+∞, σ2

2 > − µn−1 .


Gradient Flows

Let V (x) = 12αx2. The generator of the OU process can be written

as:L = −∂xV∂x + D∂2

x .

Consider diffusion processes with a potential V (x), notnecessarily quadratic:

L = −∇V (x) · ∇ + D∆. (50)

This is a gradient flow perturbed by noise whose strength isD = kBT where kB is Boltzmann’s constant and T the absolutetemperature.

The corresponding stochastic differential equation is

dXt = −∇V (Xt) dt +√

2D dWt .


The corresponding FP equation is:

∂p∂t

= ∇ · (∇Vp) + D∆p. (51)

It is not possible to calculate the time dependent solution of thisequation for an arbitrary potential. We can, however, alwayscalculate the stationary solution.


TheoremAssume that V (x) is smooth and that

e−V (x)/D ∈ L1(Rd ). (52)

Then the Markov process with generator (50) is ergodic. The uniqueinvariant distribution is the Gibbs distribution

p(x) =1Z

e−V (x)/D , (53)

where the normalization factor Z is the partition function

Z =

∫

Rde−V (x)/D dx .


The fact that the Gibbs distribution is an invariant distributionfollows by direct substitution. Uniqueness follows from a PDEsargument (see discussion below).

It is more convenient to "normalize" the solution of theFokker-Planck equation wrt the invariant distribution.

TheoremLet p(x , t) be the solution of the Fokker-Planck equation (51), assumethat (52) holds and let ρ(x) be the Gibbs distribution (187). Defineh(x , t) through

p(x , t) = h(x , t)ρ(x).

Then the function h satisfies the backward Kolmogorov equation :

∂h∂t

= −∇V · ∇h + D∆h, h(x ,0) = p(x ,0)ρ−1(x). (54)


Proof.The initial condition follows from the definition of h. We calculate thegradient and Laplacian of p:

∇p = ρ∇h − ρhD−1∇V

and

∆p = ρ∆h − 2ρD−1∇V · ∇h + hD−1∆Vρ+ h|∇V |2D−2ρ.

We substitute these formulas into the FP equation to obtain

ρ∂h∂t

= ρ(−∇V · ∇h + D∆h

),

from which the claim follows.


Consequently, in order to study properties of solutions to the FPequation, it is sufficient to study the backward equation (54).

The generator L is self-adjoint, in the right function space.

We define the weighted L2 space L2ρ:

L2ρ =

f |∫

Rd|f |2ρ(x) dx <∞

,

where ρ(x) is the Gibbs distribution. This is a Hilbert space withinner product

(f ,h)ρ =

∫

Rdfhρ(x) dx .


TheoremAssume that V (x) is a smooth potential and assume thatcondition (52) holds. Then the operator

L = −∇V (x) · ∇ + D∆

is self-adjoint in L2ρ. Furthermore, it is non-positive, its kernel consists

of constants.


Proof.

Let f , ∈ C20(Rd). We calculate

(Lf ,h)ρ =

∫

Rd(−∇V · ∇ + D∆)fhρdx

=

∫

Rd(∇V · ∇f )hρdx − D

∫

Rd∇f∇hρdx − D

∫

Rd∇fh∇ρdx

= −D∫

Rd∇f · ∇hρdx ,

from which self-adjointness follows.


If we set f = h in the above equation we get

(Lf , f )ρ = −D‖∇f‖2ρ,

which shows that L is non-positive.Clearly, constants are in the null space of L. Assume that f ∈ N (L).Then, from the above equation we get

0 = −D‖∇f‖2ρ,

and, consequently, f is a constant.Remark

The expression (−Lf , f )ρ is called the Dirichlet form of theoperator L. In the case of a gradient flow, it takes the form

(−Lf , f )ρ = D‖∇f‖2ρ. (55)


Using the properties of the generator L we can show that thesolution of the Fokker-Planck equation converges to the Gibbsdistribution exponentially fast.

For this we need the following result.

TheoremAssume that the potential V satisfies the convexity condition

D2V > λI.

Then the corresponding Gibbs measure satisfies the Poincaréinequality with constant λ:

∫

Rdfρ = 0 ⇒ ‖∇f‖ρ >

√λ‖f‖ρ. (56)


Theorem

Assume that p(x ,0) ∈ L2(eV/D). Then the solution p(x , t) of theFokker-Planck equation (51) converges to the Gibbs distributionexponentially fast:

‖p(·, t) − Z−1e−V‖ρ−1 6 e−λDt‖p(·,0) − Z−1e−V‖ρ−1 . (57)


Proof.We Use (54), (55) and (56) to calculate

− ddt

‖(h − 1)‖2ρ = −2

(∂h∂t,h − 1

)

ρ

= −2 (Lh,h − 1)ρ

= (−L(h − 1),h − 1)ρ = 2D‖∇(h − 1)‖ρ> 2Dλ‖h − 1‖2

ρ.

Our assumption on p(·,0) implies that h(·,0) ∈ L2ρ. Consequently, the

above calculation shows that

‖h(·, t) − 1‖ρ 6 e−λDt‖H(·,0) − 1‖ρ.

This, and the definition of h, p = ρh, lead to (57).


The assumption∫

Rd|p(x ,0)|2Z−1eV/D <∞

is very restrictive (think of the case where V = x2).

The function space L2(ρ−1) = L2(e−V/D) in which we proveconvergence is not the right space to use. Since p(·, t) ∈ L1,ideally we would like to prove exponentially fast convergence in L1.

We can prove convergence in L1 using the theory of logarithmicSobolev inequalities . In fact, we can also prove convergence inrelative entropy :

H(p|ρV ) :=

∫

Rdp ln

(pρV

)dx .


The relative entropy norm controls the L1 norm:

‖ρ1 − ρ2‖L1 6 CH(ρ1|ρ2)

Using a logarithmic Sobolev inequality, we can prove exponentiallyfast convergence to equilibrium, assuming only that the relativeentropy of the initial conditions is finite.

TheoremLet p denote the solution of the Fokker–Planck equation (51) wherethe potential is smooth and uniformly convex. Assume that the theinitial conditions satisfy

H(p(·,0)|ρV ) <∞.

Then p converges to the Gibbs distribution exponentially fast in relativeentropy:

H(p(·, t)|ρV ) 6 e−λDtH(p(·,0)|ρV ).


Convergence to equilibrium for kinetic equations , both linear andnon-linear (e.g., the Boltzmann equation) has been studiedextensively.

It has been recognized that the relative entropy plays a veryimportant role.

For more information see

On the trend to equilibrium for the Fokker-Planck equation: aninterplay between physics and functional analysis by P.A.Markowich and C. Villani, 1999.


Eigenfunction Expansions

Consider the generator of a gradient flow with a uniformly convexpotential

L = −∇V · ∇ + D∆. (58)

We know that1 L is a non-positive self-adjoint operator on L2

ρ.

2 It has a spectral gap:

(Lf , f )ρ 6 −Dλ‖f‖2ρ

where λ is the Poincaré constant of the potential V .

The above imply that we can study the spectral problem for −L:

−Lfn = λnfn, n = 0,1, . . .


The operator −L has real, discrete spectrum with

0 = λ0 < λ1 < λ2 < . . .

Furthermore, the eigenfunctions fj∞j=1 form an orthonormal basis

in L2ρ: we can express every element of L2

ρ in the form of ageneralized Fourier series:

φ = φ0 +

∞∑

n=1

φnfn, φn = (φ, fn)ρ (59)

with (fn, fm)ρ = δnm and φ0 ∈ N (L).

This enables us to solve the time dependent Fokker–Planckequation in terms of an eigenfunction expansion.

Consider the backward Kolmogorov equation (54).

We assume that the initial conditions h0(x) = φ(x) ∈ L2ρ and

consequently we can expand it in the form (59).


We look for a solution of (54) in the form

h(x , t) =∞∑

n=0

hn(t)fn(x).

We substitute this expansion into the backward Kolmogorovequation:

∂h∂t

=

∞∑

n=0

hnfn = L( ∞∑

n=0

hnfn

)(60)

=

∞∑

n=0

−λnhnfn. (61)

We multiply this equation by fm, integrate wrt the Gibbs measureand use the orthonormality of the eigenfunctions to obtain thesequence of equations

hn = −λnhn, n = 0,1, . . .


The solution is

h0(t) = φ0, hn(t) = e−λntφn, n = 1,2, . . .

Notice that

1 =

∫

Rdp(x ,0) dx =

∫

Rdp(x , t) dx

=

∫

Rdh(x , t)Z−1e−βV dx = (h,1)ρ = (φ,1)ρ

= φ0.


Consequently, the solution of the backward Kolmogorov equationis

h(x , t) = 1 +

∞∑

n=1

e−λntφnfn.

This expansion, together with the fact that all eigenvalues arepositive (n > 1), shows that the solution of the backwardKolmogorov equation converges to 1 exponentially fast.

The solution of the Fokker–Planck equation is

p(x , t) = Z−1e−βV (x)

(

1 +

∞∑

n=1

e−λntφnfn

)

.


Self-adjointness

The Fokker–Planck operator of a general diffusion process is notself-adjoint in general.

In fact, it is self-adjoint if and only if the drift term is the gradientof a potential (Nelson, ≈ 1960). This is also true in infinitedimensions (Stochastic PDEs).

Markov Processes whose generator is a self-adjoint operator arecalled reversible : for all t ∈ [0,T ] Xt and XT−t have the sametransition probability (when Xt is statinary).

Reversibility is equivalent to the invariant measure being a Gibbsmeasure.

See Thermodynamics of the general duffusion process:time-reversibility and entropy production, H. Qian, M. Qian, X.Tang, J. Stat. Phys., 107, (5/6), 2002, pp. 1129–1141.


Reduction to a Schrödinger Equation

Lemma

The Fokker–Planck operator for a gradient flow can be written in theself-adjoint form

∂p∂t

= D∇ ·(

e−V/D∇(

eV/Dp))

. (62)

Define now ψ(x , t) = eV/2Dp(x , t). Then ψ solves the PDE

∂ψ

∂t= D∆ψ − U(x)ψ, U(x) :=

|∇V |24D

− ∆V2. (63)

Let H := −D∆ + U. Then L∗ and H have the same eigenvalues. Thenth eigenfunction φn of L∗ and the nth eigenfunction ψn of H areassociated through the transformation

ψn(x) = φn(x) exp(

V (x)

2D

).


Remarks

1 From equation (62) shows that the FP operator can be written inthe form

L∗· = D∇ ·(

e−V/D∇(

eV/D·))

.

2 The operator that appears on the right hand side of eqn. (63) hasthe form of a Schrödinger operator :

−H = −D∆ + U(x).

3 The spectral problem for the FP operator can be transformed intothe spectral problem for a Schrödinger operator. We can thus useall the available results from quantum mechanics to study the FPequation and the associated SDE.

4 In particular, the weak noise asymptotics D ≪ 1 is equivalent tothe semiclassical approximation from quantum mechanics.


Proof.

We calculate

D∇ ·(

e−V/D∇(

eV/Df))

= D∇ ·(

e−V/D(

D−1∇Vf + ∇f)

eV/D)

= ∇ · (∇Vf + D∇f ) = L∗f .

Consider now the eigenvalue problem for the FP operator:

−L∗φn = λnφn.

Set φn = ψn exp(− 1

2D V). We calculate −L∗φn:


−L∗φn = −D∇ ·(

e−V/D∇(

eV/Dψne−V/2D))

= −D∇ ·(

e−V/D(∇ψn +

∇V2D

ψn

)eV/2D

)

=

(−D∆ψn +

(−|∇V |2

4D+

∆V2D

)ψn

)e−V/2D

= e−V/2DHψn.

From this we conclude that e−V/2DHψn = λnψne−V/2D from which theequivalence between the two eigenvalue problems follows.


Remarks

1 We can rewrite the Schrödinger operator in the form

H = DA∗A, A = ∇ +∇U2D

, A∗ = −∇ +∇U2D

.

2 These are creation and annihilation operators. They can also bewritten in the form

A· = e−U/2D∇(

eU/2D·), A∗· = eU/2D∇

(e−U/2D·

)

3 The forward the backward Kolmogorov operators have the sameeigenvalues. Their eigenfunctions are related through

φBn = φF

n exp (−V/D) ,

where φBn and φF

n denote the eigenfunctions of the backward andforward operators, respectively.


The OU Process and Hermite Polynomials

The generator of the OU process is

L = −yddy

+ Dd2

dy2 (64)

The OU process is an ergodic Markov process whose uniqueinvariant measure is absolutely continuous with respect to theLebesgue measure on R with density ρ(y) ∈ C∞(R)

ρ(y) =1√2πD

e− y2

2D .

Let L2ρ denote the closure of C∞ functions with respect to the norm

‖f‖2ρ =

∫

R

f (y)2ρ(y) dy .

The space L2ρ is a Hilbert space with inner product

(f ,g)ρ =

∫

R

f (y)g(y)ρ(y) dy .


Consider the eigenvalue problem for L:

−Lfn = λnfn.

The operator L has discrete spectrum in L2ρ:

λn = n, n ∈ N .

The corresponding eigenfunctions are the normalized Hermitepolynomials:

fn(y) =1√n!

Hn

(y√D

), (65)

where

Hn(y) = (−1)ney2

2dn

dyn

(e− y2

2

).


The first few Hermite polynomials are:

H0(y) = 1,

H1(y) = y ,

H2(y) = y2 − 1,

H3(y) = y3 − 3y ,

H4(y) = y4 − 3y2 + 3,

H5(y) = y5 − 10y3 + 15y .

Hn is a polynomial of degree n.

Only odd (even) powers appear in Hn when n is odd (even).


Lemma

The eigenfunctions fn(y)∞n=1 of the generator of the OU process Lsatisfy the following properties.

1 They form an orthonormal set in L2ρ:

(fn, fm)ρ = δnm.

2 fn(y) : n ∈ N is an orthonormal basis in L2ρ.

3 Define the creation and annihilation operators on C1(R) by

A+φ(y) =1√

D(n + 1)

(−D

dφdy

(y) + yφ(y)

), y ∈ R

and

A−φ(y) =

√Dn

dφdy

(y).

ThenA+fn = fn+1 and A−fn = fn−1. (66)


4. The eigenfunctions fn satisfy the following recurrence relation

yfn(y) =√

Dnfn−1(y) +√

D(n + 1)fn+1(y), n > 1. (67)

5. The functionH(y ;λ) = eλy−λ2

2

is a generating function for the Hermite polynomials. In particular

H(

y√D

;λ

)=

∞∑

n=0

λn√

n!fn(y), λ ∈ C, y ∈ R.


The Klein-Kramers-Chandrasekhar Equation

Consider a diffusion process in two dimensions for the variables q(position) and momentum p. The generator of this Markov processis

L = p · ∇q −∇qV∇p + γ(−p∇p + D∆p). (68)

The L2(dpdq)-adjoint is

L∗ρ = −p · ∇qρ−∇qV · ∇pρ+ γ (∇p(pρ) + D∆pρ) .

The corresponding FP equation is:

∂p∂t

= L∗p.

The corresponding stochastic differential equations is theLangevin equation

Xt = −∇V (Xt) − γXt +√

2γDWt . (69)

This is Newton’s equation perturbed by dissipation and noise.


The Klein-Kramers-Chandrasekhar equation was first derived byKramers in 1923 and was studied by Kramers in his famous paper"Brownian motion in a field of force and the diffusion model ofchemical reactions", Physica 7(1940), pp. 284-304.

Notice that L∗ is not a uniformly elliptic operator: there are secondorder derivatives only with respect to p and not q. This is anexample of a degenerate elliptic operator. It is, however,hypoelliptic : we can still prove existence and uniqueness ofsolutions for the FP equation, and obtain estimates on thesolution.

It is not possible to obtain the solution of the FP equation for anarbitrary potential.

We can calculate the (unique normalized) solution of thestationary Fokker-Planck equation.


TheoremAssume that V (x) is smooth and that

e−V (x)/D ∈ L1(Rd ).

Then the Markov process with generator (140) is ergodic. The uniqueinvariant distribution is the Maxwell-Boltzmann distribution

ρβ(p,q) =1Z

e−βH(p,q) (70)

where

H(p,q) =12‖p‖2 + V (q)

is the Hamiltonian , β = (kBT )−1 is the inverse temperature and thenormalization factor Z is the partition function

Z =

∫

R2de−βH(p,q) dpdq.


It is possible to obtain rates of convergence in either a weightedL2-norm or the relative entropy norm.

H(p(·, t)|ρ) 6 Ce−αt .

The proof of this result is quite complicated, since the generator Lis degenerate and non-selfadjoint.

See F. Herau and F. Nier, Isotropic hypoellipticity and trend toequilibrium for the Fokker-Planck equation with a high-degreepotential, Arch. Ration. Mech. Anal., 171(2),(2004), 151–218.

See also C. Villani, Hypocoercivity, AMS 2008.


Consider a second order elliptic operator of the form

L = X0 +

d∑

i=1

X ∗i Xi ,

where Xjdj=0 are vector fields (first order differential operators).

Define the Lie algebras

A0 = Lie(X1,X1, . . .Xd),

A1 = Lie([X0,X1], [X0,X2], . . . [X0,Xd ]),

. . . = . . . ,

Ak = Lie([X0,X ]; X ∈ Ak−1), k > 1

DefineH = Lie(A0,A1, . . . )

Hörmander’s Hypothesis. H is full in the sense that, for every x ∈ Rd

the vectors H(x) : H ∈ H span TxRd :

spanH(x); H ∈ H = TxRd .


Theorem (Hörmander/Kolmogorov.)Under the above hypothesis, the diffusion process Xt with generator Lhas a transition density

(Pt f )(x) =

∫

Rdp(t , x , y)f (y) dy ,

where p(·, ·, ·) is a smooth function on (0,+∞) × Rd × R

d . Moreover,the function p satisfies Kolmogorov’s forward and backward equations

∂

∂tp(·, x , ·) = L∗

yp(·, x , ·), x ∈ Rd (71)

and∂

∂tp(·, ·, y) = Lxp(·, ·, y), y ∈ R

d . (72)

For every x ∈ Rd the function p(·, x , ·) is the fundamental solution

(Green’s function) of (71), such that, for each f ∈ C0(Rd),

limt→0∫

Rd p(t , x , y)f (y) dy = f (x).


ExampleConsider the SDE

x =√

2W .

Write it as a first order system

x = y , y =√

2W .

The generator of this process is

L = X0 + X 21 , X0 = y∂x , X1 = ∂y .

Consequently [X0,X1] = −∂x and

Lie(X1, [X0,X1]) = Lie(−∂x , ∂y ),

which spans TxR2.


Let ρ(q,p, t) be the solution of the Kramers equation and let ρβ(q,p)be the Maxwell-Boltzmann distribution. We can write

ρ(q,p, t) = h(q,p, t)ρβ(q,p),

where h(q,p, t) solves the equation

∂h∂t

= −Ah + γSh

whereA = p · ∇q −∇qV · ∇p, S = −p · ∇p + β−1∆p.

The operator A is antisymmetric in L2ρ := L2(R2d ; ρβ(q,p)), whereas S

is symmetric.


Let Xi := − ∂∂pi

. The L2ρ-adjoint of Xi is

X ∗i = −βpi +

∂

∂pi.

We have that

S = β−1d∑

i=1

X ∗i Xi .

Consequently, the generator of the Markov process q(t), p(t) can bewritten in Hörmander’s "sum of squares" form:

L = A + γβ−1d∑

i=1

X ∗i Xi . (73)

We calculate the commutators between the vector fields in (73):

[A,Xi ] =∂

∂qi, [Xi ,Xj ] = 0, [Xi ,X

∗j ] = βδij .


Consequently,

Lie(X1, . . .Xd , [A,X1], . . . [A,Xd ]) = Lie(∇p,∇q)

which spans Tp,qR2d for all p, q ∈ R

d . This shows that the generator Lis a hypoelliptic operator.Let now Yi = − ∂

∂piwith L2

ρ-adjoint Y ∗i = ∂

∂qi− β ∂V

∂qi. We have that

X ∗i Yi − Y ∗

i Xi = β

(pi

∂

∂qi− ∂V∂qi

∂

∂pi

).

Consequently, the generator can be written in the form

L = β−1d∑

i=1

(X ∗i Yi − Y ∗

i Xi + γX ∗i Xi) .

Notice also that

LV := −∇qV∇q + β−1∆q = β−1d∑

i=1

Y ∗i Yi .


The phase-space Fokker-Planck equation can be written in theform

∂ρ

∂t+ p · ∇qρ−∇qV · ∇pρ = Q(ρ, fB)

where the collision operator has the form

Q(ρ, fB) = D∇ ·(

fB∇(

f−1B ρ

)).

The Fokker-Planck equation has a similar structure to theBoltzmann equation (the basic equation in the kinetic theory ofgases), with the difference that the collision operator for the FPequation is linear.

Convergence of solutions of the Boltzmann equation to theMaxwell-Boltzmann distribution has also been proved. See

L. Desvillettes and C. Villani: On the trend to global equilibrium forspatially inhomogeneous kinetic systems: the Boltzmannequation. Invent. Math. 159, 2 (2005), 245-316.


We can study the backward and forward Kolmogorov equationsfor (183) by expanding the solution with respect to the Hermitebasis.

We consider the problem in 1d. We set D = 1. The generator ofthe process is:

L = p∂q − V ′(q)∂p + γ(−p∂p + ∂2

p

).

=: L1 + γL0,

where

L0 := −p∂p + ∂2p and L1 := p∂q − V ′(q)∂p.

The backward Kolmogorov equation is

∂h∂t

= Lh. (74)


The solution should be an element of the weighted L2-space

L2ρ =

f |∫

R2|f |2Z−1e−βH(p,q) dpdq <∞

.

We notice that the invariant measure of our Markov process is aproduct measure:

e−βH(p,q) = e−β 12 |p|

2e−βV (q).

The space L2(e−β 12 |p|

2dp) is spanned by the Hermite polynomials.

Consequently, we can expand the solution of (74) into the basis ofHermite basis:

h(p,q, t) =

∞∑

n=0

hn(q, t)fn(p), (75)

where fn(p) = 1/√

n!Hn(p).


Our plan is to substitute (75) into (74) and obtain a sequence ofequations for the coefficients hn(q, t).

We have:

L0h = L0

∞∑

n=0

hnfn = −∞∑

n=0

nhnfn

FurthermoreL1h = −∂qV∂ph + p∂qh.

We calculate each term on the right hand side of the aboveequation separately. For this we will need the formulas

∂pfn =√

nfn−1 and pfn =√

nfn−1 +√

n + 1fn+1.


p∂qh = p∂q

∞∑

n=0

hnfn = p∂ph0 +

∞∑

n=1

∂qhnpfn

= ∂qh0f1 +

∞∑

n=1

∂qhn

(√nfn−1 +

√n + 1fn+1

)

=∞∑

n=0

(√

n + 1∂qhn+1 +√

n∂qhn−1)fn

with h−1 ≡ 0.Furthermore

∂qV∂ph =∞∑

n=0

∂qVhn∂pfn =∞∑

n=0

∂qVhn√

nfn−1

=

∞∑

n=0

∂qVhn+1√

n + 1fn.


Consequently:

Lh = L1 + γL1h

=

∞∑

n=0

(− γnhn +

√n + 1∂qhn+1

+√

n∂qhn−1 +√

n + 1∂qVhn+1)fn

Using the orthonormality of the eigenfunctions of L0 we obtain thefollowing set of equations which determine hn(q, t)∞n=0.

hn = −γnhn +√

n + 1∂qhn+1

+√

n∂qhn−1 +√

n + 1∂qVhn+1, n = 0,1, . . .

This is set of equations is usually called the Brinkman hierarchy(1956).


We can use this approach to develop a numerical method forsolving the Klein-Kramers equation.

For this we need to expand each coefficient hn in an appropriatebasis with respect to q.

Obvious choices are other the Hermite basis (polynomialpotentials) or the standard Fourier basis (periodic potentials).

We will do this for the case of periodic potentials.

The resulting method is usually called the continued fractionexpansion . See Risken (1989).


The Hermite expansion of the distribution function wrt to thevelocity is used in the study of various kinetic equations (includingthe Boltzmann equation). It was initiated by Grad in the late 40’s.

It quite often used in the approximate calculation of transportcoefficients (e.g. diffusion coefficient).

This expansion can be justified rigorously for the Fokker-Planckequation. See

J. Meyer and J. Schröter, Comments on the Grad Procedure forthe Fokker-Planck Equation, J. Stat. Phys. 32(1) pp.53-69 (1983).

This expansion can also be used in order to solve the Poissonequation −Lφ = f (p,q). See G.A. Pavliotis and T. VogiannouDiffusive Transport in Periodic Potentials: Underdampeddynamics, Fluct. Noise Lett., 8(2) L155-L173 (2008).


There are very few potentials for which we can calculate theeigenvalues and eigenfunctions of the generator of the Markovprocess q(t), p(t).

We can calculate everything explicitly for the quadratic (harmonic)potential

V (q) =12ω2

0q2. (76)

The Langevin equation is

q = −ω20q − γq +

√2γβ−1W (77)

orq = p, p = −ω2

0q − γp +√

2γβ−1W . (78)

This is a linear equation that can be solved explicitly (The solutionis a Gaussian stochastic process).


The generator of the process q(t), p(t)

L = p∂q − ω20q∂p + γ(−p∂p + β−1∂2

p). (79)

The Fokker-Planck operator is

L = p∂q − ω20q∂p + γ(−p∂p + β−1∂2

p). (80)

The process q(t), p(t) is an ergodic Markov process withGaussian invariant measure

ρβ(q,p) dqdp =βω0

2πe−β

2 p2−βω20

2 q2. (81)


For the calculation of the eigenvalues and eigenfunctions of theoperator L it is convenient to introduce creation and annihilationoperator in both the position and momentum variables:

a− = β−1/2∂p, a+ = −β−1/2∂p + β1/2p (82)

and

b− = ω−10 β−1/2∂q , b+ = −ω−1

0 β−1/2∂q + ω0β1/2p. (83)

We have thata+a− = −β−1∂2

p + p∂p

andb+b− = −β−1∂2

q + q∂q

Consequently, the operator

L = −a+a− − b+b− (84)

is the generator of the OU process in two dimensions.


Exercises

1 Calculate the eigenvalues and eigenfunctions of L. Show thatthere exists a transformation that transforms L into theSchrödinger operator of the two-dimensional quantum harmonicoscillator.

2 Show that the operators a±, b± satisfy the commutation relations

[a+,a−] = −1, (85a)

[b+,b−] = −1, (85b)

[a±,b±] = 0. (85c)


Using the operators a± and b± we can write the generator L in theform

L = −γa+a− − ω0(b+a− − a+b−), (86)

We want to find first order differential operators c± and d± so thatthe operator (86) becomes

L = −Cc+c− − Dd+d− (87)

for some appropriate constants C and D.

Our goal is, essentially, to map L to the two-dimensional OUprocess.

We require that that the operators c± and d± satisfy thecanonical commutation relations

[c+, c−] = −1, (88a)

[d+,d−] = −1, (88b)

[c±,d±] = 0. (88c)


The operators c± and d± should be given as linear combinationsof the old operators a± and b±. They should be of the form

c+ = α11a+ + α12b+, (89a)

c− = α21a− + α22b−, (89b)

d+ = β11a+ + β12b+, (89c)

d− = β21a− + β22b−. (89d)

Notice that the c− and d− are not the adjoints of c+ and d+.

If we substitute now these equations into (87) and equate itwith (86) and into the commutation relations (88) we obtain asystem of equations for the coefficients αij, βij.


In order to write down the formulas for these coefficients it isconvenient to introduce the eigenvalues of the deterministicproblem

q = −γq − ω20q.


q(t) = C1e−λ1t + C2e−λ2t

with

λ1,2 =γ ± δ

2, δ =

√γ2 − 4ω2

0. (90)

The eigenvalues satisfy the relations

λ1 + λ2 = γ, λ1 − λ2 = δ, λ1λ2 = ω20. (91)


LemmaLet L be the generator (86) and let c±, d± be the operators

c+ =1√δ

(√λ1a+ +

√λ2b+

), c− =

1√δ

(√λ1a− −

√λ2b−

),

(92a)

d+ =1√δ

(√λ2a+ +

√λ1b+

), d− =

1√δ

(−√λ2a− +

√λ1b−

).

(92b)Then c±, d± satisfy the canonical commutation relations (88) as wellas

[L, c±] = −λ1c±, [L,d±] = −λ2d±. (93)

Furthermore, the operator L can be written in the form

L = −λ1c+c− − λ2d+d−. (94)


Proof. first we check the commutation relations:

[c+, c−] =1δ

(λ1[a

+,a−] − λ2[b+,b−]

)

=1δ(−λ1 + λ2) = −1.

Similarly,

[d+,d−] =1δ

(−λ2[a

+,a−] + λ1[b+,b−]

)

=1δ(λ2 − λ1) = −1.

Clearly, we have that

[c+,d+] = [c−,d−] = 0.

Furthermore,

[c+,d−] =1δ

(−√λ1λ2[a

+,a−] +√λ1λ2[b

+,b−])

=1δ(−√λ1λ2 + −

√λ1λ2) = 0.


Finally:

[L, c+] = −λ1c+c−c+ + λ1c+c+c−

= −λ1c+(1 + c+c−) + λ1c+c+c−

= −λ1c+(1 + c+c−) + λ1c+c+c−

= −λ1c+,

and similarly for the other equations in (93). Now we calculate

L = −λ1c+c− − λ2d+d−

= −λ22 − λ2

1

δa+a− + 0b+b− +

√λ1λ2

δ(λ1 − λ2)a

+b−

+1δ

√λ1λ2(−λ1 + λ2)b

+a−

= −γa+a− − ω0(b+a− − a+b−),

which is precisely (86). In the above calculation we used (91).


Theorem

The eigenvalues and eigenfunctions of the generator of the Markovprocess q, p (78) are

λnm = λ1n + λ2m =12γ(n + m) +

12δ(n − m), n,m = 0, 1, . . . (95)

and

φnm(q,p) =1√

n!m!(c+)n(d+)m1, n,m = 0, 1, . . . (96)


We have

[L, (c+)2] = L(c+)2 − (c+)2L= (c+L − λ1c+)c+ − c+(Lc+ + λ1c+)

= −2λ1(c+)2

and similarly [L, (d+)2] = −2λ1(c+)2. A simple induction argumentnow shows that ( exercise)

[L, (c+)n] = −nλ1(c+)n and [L, (d+)m] = −mλ1(d

+)m. (97)

We use (97) to calculate

L(c+)n(d+)n1 = (c+)nL(d+)m1 − nλ1(c+)n(d+m)1

= (c+)n(d+)mL1 − mλ2(c+)n(d+m)1 −

nλ1(c+)n(d+m)1

= −nλ1(c+)n(d+m)1 − mλ2(c

+)n(d+m)1

from which (95) and (96) follow.Pavliotis (IC) StochProc January 16, 2011 216 / 367

Exercise1 Show that

[L, (c±)n] = −nλ1(c±)n, [L, (d±)n] = −nλ1(d

±)n, (98)

[c−, (c+)n] = n(c+)n−1, [d−, (d+)n] = n(d+)n−1. (99)

Remark In terms of the operators a±, b± the eigenfunctions of L are

φnm =√

n!m!δ−n+m

2 λn/21 λ

m/22

n∑

ℓ=0

m∑

k=0

1k!(m − k)!ℓ!(n − ℓ)!

×

(λ1

λ2

) k−ℓ2

(a+)n+m−k−ℓ(b+)ℓ+k1.


The first few eigenfunctions are

φ00 = 1.

φ10 =

√β

`√λ1p +

√λ2ω0q

´

√δ

.

φ01 =

√β

`√λ2p +

√λ1ω0q

´

√δ

φ11 =−2

√λ1

√λ2 +

√λ1β p2√λ2 + β pλ1ω0q + ω0β qλ2p +

√λ2ω0

2β q2√λ1

δ.

φ20 =−λ1 + β p2λ1 + 2

√λ2β p

√λ1ω0q − λ2 + ω0

2β q2λ2√2δ

.

φ02 =−λ2 + β p2λ2 + 2

√λ2β p

√λ1ω0q − λ1 + ω0

2β q2λ1√2δ

.


The eigenfunctions are not orthonormal.The first eigenvalue, corresponding to the constant eigenfunction,is 0:

λ00 = 0.

The operator L is not self-adjoint and consequently, we do notexpect its eigenvalues to be real.Whether the eigenvalues are real or not depends on the sign ofthe discriminant ∆ = γ2 − 4ω2

0.In the underdamped regime, γ < 2ω0 the eigenvalues arecomplex:

λnm =12γ(n + m) +

12

i√

−γ2 + 4ω20(n − m), γ < 2ω0.

This it to be expected, since the underdamped regime thedynamics is dominated by the deterministic Hamiltonian dynamicsthat give rise to the antisymmetric Liouville operator.We set ω =

√(4ω2

0 − γ2), i.e. δ = 2iω. The eigenvalues can bewritten as

λnm =γ

2(n + m) + iω(n − m).


0 0.5 1 1.5 2 2.5 3−3

−2

−1

0

1

2

3

Re (λnm)

Im (

λnm)

Figure: First few eigenvalues of L for γ = ω = 1.


In Figure 34 we present the first few eigenvalues of L in theunderdamped regime.The eigenvalues are contained in a cone on the right half of thecomplex plane. The cone is determined by

λn0 =γ

2n + iωn and λ0m =

γ

2m − iωm.

The eigenvalues along the diagonal are real:

λnn = γn.

In the overdamped regime, γ > 2ω0 all eigenvalues are real:

λnm =12γ(n + m) +

12

√γ2 − 4ω2

0(n − m), γ > 2ω0.

In fact, in the overdamped limit γ → +∞, the eigenvalues of thegenerator L converge to the eigenvalues of the generator of theOU process:

λnm = γn +ω2

0

γ(n − m) + O(γ−3).


The eigenfunctions of L do not form an orthonormal basis inL2β := L2(R2,Z−1e−βH) since L is not a selfadjoint operator.

Using the eigenfunctions/eigenvalues of L we can easily calculatethe eigenfunctions/eigenvalues of the L2

β adjoint of L.

The adjoint operator is

L := −A + γS

= −ω0(b+a− − b−a+) + γa+a−

= −λ1(c−)∗(c+)∗ − λ2(d

−) ∗ (d+)∗,

where

(c+)∗ =1√δ

(√λ1a− +

√λ2b−

), (c−)∗ =

1√δ

(√λ1a+ −

√λ2b+

)

(100a)

(d+)∗ =1√δ

(√λ2a− +

√λ1b−

), (d−)∗ =

1√δ

(−√λ2a+ +

√λ1b+

(100b)


Exercise1 Show by direct substitution that L can be written in the form

L = −λ1(c−)∗(c+)∗ − λ2(d

−)∗(d+)∗.

2 Calculate the commutators

[(c+)∗, (c−)∗], [(d+)∗, (d−)∗], [(c±)∗, (d±)∗], [L, (c±)∗], [L, (d±)∗].

1 L has the same eigenvalues as L:

−Lψnm = λnmψnm,

2 where λnm are given by (95).3 The eigenfunctions are

ψnm =1√

n!m!((c−)∗)n((d−)∗)m1. (101)


Theorem

The eigenfunctions of L and L satisfy the biorthonormality relation∫ ∫

φnmψℓkρβ dpdq = δnℓδmk . (102)

Proof. We will use formulas (98). Notice that using the third and fourthof these equations together with the fact that c−1 = d−1 = 0 we canconclude that (for n > ℓ)

(c−)ℓ(c+)n1 = n(n − 1) . . . (n − ℓ+ 1)(c+)n−ℓ. (103)We have

Z Z

φnmψℓkρβ dpdq =1√

n!m!ℓ!k !

Z Z

((c+))n((d+))m1((c−)∗)ℓ((d−)∗)k 1ρβ dpdq

=n(n − 1) . . . (n − ℓ+ 1)m(m − 1) . . . (m − k + 1)√

n!m!ℓ!k !×

Z Z

((c+))n−ℓ((d+))m−k 1ρβ dpdq

= δnℓδmk ,

since all eigenfunctions average to 0 with respect to ρβ .Pavliotis (IC) StochProc January 16, 2011 224 / 367

From the eigenfunctions of L we can obtain the eigenfunctions ofthe Fokker-Planck operator. Using the formula

L∗(fρβ) = ρLf

we immediately conclude that the the Fokker-Planck operator hasthe same eigenvalues as those of L and L. The eigenfunctionsare

ψ∗nm = ρβφnm = ρβ

1√n!m!

((c−)∗)n((d−)∗)m1. (104)


STOCHASTIC DIFFERENTIAL EQUATIONS (SDEs)


In this part of the course we will study stochastic differentialequation (SDEs): ODEs driven by Gaussian white noise.

Let W (t) denote a standard m–dimensional Brownian motion,h : Z → R

d a smooth vector-valued function and γ : Z → Rd×m a

smooth matrix valued function (in this course we will takeZ = T

d , Rd or R

l ⊕ Td−l .

Consider the SDE

dzdt

= h(z) + γ(z)dWdt

, z(0) = z0. (105)

We think of the term dWdt as representing Gaussian white noise: a

mean-zero Gaussian process with correlation δ(t − s)I.

The function h in (105) is sometimes referred to as the drift and γas the diffusion coefficient.


Such a process exists only as a distribution. The preciseinterpretation of (105) is as an integral equation forz(t) ∈ C(R+,Z):

z(t) = z0 +

∫ t

0h(z(s))ds +

∫ t

0γ(z(s))dW (s). (106)

In order to make sense of this equation we need to define thestochastic integral against W (s).


The Itô Stochastic Integral

For the rigorous analysis of stochastic differential equations it isnecessary to define stochastic integrals of the form

I(t) =

∫ t

0f (s) dW (s), (107)

where W (t) is a standard one dimensional Brownian motion. Thisis not straightforward because W (t) does not have boundedvariation.

In order to define the stochastic integral we assume that f (t) is arandom process, adapted to the filtration Ft generated by theprocess W (t), and such that

E

(∫ T

0f (s)2 ds

)<∞.


The Itô stochastic integral I(t) is defined as the L2–limit of theRiemann sum approximation of (107):

I(t) := limK→∞

K−1∑

k=1

f (tk−1) (W (tk ) − W (tk−1)) , (108)

where tk = k∆t and K∆t = t .

Notice that the function f (t) is evaluated at the left end of eachinterval [tn−1, tn] in (108).

The resulting Itô stochastic integral I(t) is a.s. continuous in t .

These ideas are readily generalized to the case where W (s) is astandard d dimensional Brownian motion and f (s) ∈ R

m×d foreach s.


The resulting integral satisfies the Itô isometry

E|I(t)|2 =

∫ t

0E|f (s)|2F ds, (109)

where | · |F denotes the Frobenius norm |A|F =√

tr(AT A).

The Itô stochastic integral is a martingale:

EI(t) = 0

andE[I(t)|Fs] = I(s) ∀ t > s,

where Fs denotes the filtration generated by W (s).


ExampleConsider the Itô stochastic integral

I(t) =

∫ t

0f (s) dW (s),

where f ,W are scalar–valued. This is a martingale with quadraticvariation

〈I〉t =

∫ t

0(f (s))2 ds.

More generally, for f , W in arbitrary finite dimensions, the integralI(t) is a martingale with quadratic variation

〈I〉t =

∫ t

0(f (s) ⊗ f (s)) ds.


The Stratonovich Stochastic Integral

In addition to the Itô stochastic integral, we can also define theStratonovich stochastic integral. It is defined as the L2–limit of adifferent Riemann sum approximation of (107), namely

Istrat (t) := limK→∞

K−1∑

k=1

12

(f (tk−1) + f (tk )

)(W (tk ) − W (tk−1)) , (110)

where tk = k∆t and K∆t = t . Notice that the function f (t) isevaluated at both endpoints of each interval [tn−1, tn] in (110).

The multidimensional Stratonovich integral is defined in a similarway. The resulting integral is written as

Istrat (t) =

∫ t

0f (s) dW (s).


The limit in (110) gives rise to an integral which differs from the Itôintegral.

The situation is more complex than that arising in the standardtheory of Riemann integration for functions of bounded variation:in that case the points in [tk−1, tk ] where the integrand is evaluateddo not effect the definition of the integral, via a limiting process.

In the case of integration against Brownian motion, which does nothave bounded variation, the limits differ.

When f and W are correlated through an SDE, then a formulaexists to convert between them.


Existence and Uniqueness of solutions for SDEs

DefinitionBy a solution of (105) we mean a Z-valued stochastic process z(t)on t ∈ [0,T ] with the properties:

1 z(t) is continuous and Ft−adapted, where the filtration isgenerated by the Brownian motion W (t);

2 h(z(t)) ∈ L1((0,T )), γ(z(t)) ∈ L2((0,T ));

3 equation (105) holds for every t ∈ [0,T ] with probability 1.

The solution is called unique if any two solutions xi(t), i = 1,2 satisfy

P(x1(t) = x2(t), ∀t ∈ [0.T ]) = 1.


It is well known that existence and uniqueness of solutions forODEs (i.e. when γ ≡ 0 in (105)) holds for globally Lipschitz vectorfields h(x).

A very similar theorem holds when γ 6= 0.

As for ODEs the conditions can be weakened, when a prioribounds on the solution can be found.


Theorem

Assume that both h(·) and γ(·) are globally Lipschitz on Z and that z0

is a random variable independent of the Brownian motion W (t) with

E|z0|2 <∞.

Then the SDE (105) has a unique solution z(t) ∈ C(R+;Z) with

E

[∫ T

0|z(t)|2 dt

]<∞ ∀T <∞.

Furthermore, the solution of the SDE is a Markov process.


Remarks

The Stratonovich analogue of (105) is

dzdt

= h(z) + γ(z) dWdt

, z(0) = z0. (111)

By this we mean that z ∈ C(R+,Z) satisfies the integral equation

z(t) = z(0) +

∫ t

0h(z(s))ds +

∫ t

0γ(z(s)) dW (s). (112)

By using definitions (108) and (110) it can be shown that zsatisfying the Stratonovich SDE (111) also satisfies the Itô SDE

dzdt

= h(z) +12∇ ·(γ(z)γ(z)T )− 1

2γ(z)∇ ·

(γ(z)T )+ γ(z)

dWdt

,

(113a)z(0) = z0, (113b)

provided that γ(z) is differentiable.


White noise is, in most applications, an idealization of a stationaryrandom process with short correlation time. In this context theStratonovich interpretation of an SDE is particularly importantbecause it often arises as the limit obtained by using smoothapproximations to white noise.

On the other hand the martingale machinery which comes withthe Itô integral makes it more important as a mathematical object.

It is very useful that we can convert from the Itô to theStratonovich interpretation of the stochastic integral.

There are other interpretations of the stochastic integral, e.g. theKlimontovich stochastic integral.


The Definition of Brownian motion implies the scaling property

W (ct) =√

cW (t),

where the above should be interpreted as holding in law. Fromthis it follows that, if s = ct , then

dWds

=1√c

dWdt

,

again in law.

Hence, if we scale time to s = ct in (105), then we get the equation

dzds

=1c

h(z) +1√cγ(z)

dWds

, z(0) = z0.


The Stratonovich Stochastic Integral: A firstapplication of multiscale methods

When white noise is approximated by a smooth process this oftenleads to Stratonovich interpretations of stochastic integrals, atleast in one dimension.

We use multiscale analysis (singular perturbation theory forMarkov processes) to illustrate this phenomenon in aone-dimensional example.

Consider the equations

dxdt

= h(x) +1ε

f (x)y , (114a)

dydt

= −αyε2 +

√2Dε2

dVdt, (114b)

with V being a standard one-dimensional Brownian motion.


We say that the process x(t) is driven by colored noise : thenoise that appears in (114a) has non-zero correlation time. Thecorrelation function of the colored noise η(t) := y(t)/ε is (we takey(0) = 0)

R(t) = E (η(t)η(s)) =1ε2

Dα

e− α

ε2 |t−s|.

The power spectrum of the colored noise η(t) is:

f ε(x) =1ε2

Dε−2

π

1x2 + (αε−2)2

=Dπ

1ε4x2 + α2 → D

πα2

and, consequently,

limε→0

E

(y(t)ε

y(s)

ε

)=

2Dα2 δ(t − s),

which implies the heuristic

limε→0

y(t)ε

=

√2Dα2

dVdt. (115)


Another way of seeing this is by solving (114b) for y/ε:

yε

=

√2Dα2

dVdt

− ε

α

dydt. (116)

If we neglect the O(ε) term on the right hand side then we arrive,again, at the heuristic (115).

Both of these arguments lead us to conjecture the limiting Itô SDE:

dXdt

= h(X ) +

√2Dα

f (X )dVdt. (117)

In fact, as applied, the heuristic gives the incorrect limit.


whenever white noise is approximated by a smooth process, thelimiting equation should be interpreted in the Stratonovich sense,giving

dXdt

= h(X ) +

√2Dα

f (X ) dVdt. (118)

This is usually called the Wong-Zakai theorem. A similar result istrue in arbitrary finite and even infinite dimensions.

We will show this using singular perturbation theory.

Theorem

Assume that the initial conditions for y(t) are stationary and that thefunction f is smooth. Then the solution of eqn (114a) converges, in thelimit as ε→ 0 to the solution of the Stratonovich SDE (118).


Remarks

1 It is possible to prove pathwise convergence under very mildassumptions.

2 The generator of a Stratonovich SDE has the from

Lstrat = h(x)∂x +Dα

f (x)∂x (f (x)∂x ) .

3 Consequently, the Fokker-Planck operator of the StratonovichSDE can be written in divergence form:

L∗strat · = −∂x (h(x)·) +

Dα∂x

(f 2(x)∂x ·

).


4. In most applications in physics the white noise is an approximationof a more complicated noise processes with non-zero correlationtime. Hence, the physically correct interpretation of the stochasticintegral is the Stratonovich one.

5. In higher dimensions an additional drift term might appear due tothe noncommutativity of the row vectors of the diffusion matrix. Thisis related to the Lévy area correction in the theory of rough paths.


Proof of Proposition 61 The generator of the process (x(t), y(t)) is

L =1ε2

(−αy∂y + D∂2

y

)+

1ε

f (x)y∂x + h(x)∂x

=:1ε2L0 +

1εL1 + L2.

The "fast" process is an stationary Markov process with invariantdensity

ρ(y) =

√α

2πDe−αy2

2D . (119)

The backward Kolmogorov equation is

∂uε

∂t=

(1ε2L0 +

1εL1 + L2

)uε. (120)

We look for a solution to this equation in the form of a power seriesexpansion in ε:

uε(x , y , t) = u0 + εu1 + ε2u2 + . . .


We substitute this into (120) and equate terms of the same power in εto obtain the following hierarchy of equations:

−L0u0 = 0,

−L0u1 = L1u0,

−L0u2 = L1u1 + L2u0 −∂u0

∂t.

The ergodicity of the fast process implies that the null space of thegenerator L0 consists only of constant in y . Hence:

u0 = u(x , t).

The second equation in the hierarchy becomes

−L0u1 = f (x)y∂x u.


This equation is solvable since the right hand side is orthogonal to thenull space of the adjoint of L0 (this is the Fredholm alterantive ). Wesolve it using separation of variables:

u1(x , y , t) =1α

f (x)∂xuy + ψ1(x , t).

In order for the third equation to have a solution we need to requirethat the right hand side is orthogonal to the null space of L∗

0:

∫

R

(L1u1 + L2u0 −

∂u0

∂t

)ρ(y) dy = 0.

We calculate: ∫

R

∂u0

∂tρ(y) dy =

∂u∂t.

Furthermore:


∫

R

L2u0ρ(y) dy = h(x)∂x u.

Finally∫

R

L1u1ρ(y) dy =

∫

R

f (x)y∂x

(1α

f (x)∂xuy + ψ1(x , t))ρ(y) dy

=1α

f (x)∂x (f (x)∂xu) 〈y2〉 + f (x)∂xψ1(x , t)〈y〉

=Dα2 f (x)∂x (f (x)∂xu)

=Dα2 f (x)∂x f (x)∂x u +

Dα2 f (x)2∂2

x u.

Putting everything together we obtain the limiting backwardKolmogorov equation

∂u∂t

=

(h(x) +

Dα2 f (x)∂x f (x)

)∂xu +

Dα2 f (x)2∂2

x u,

from which we read off the limiting Stratonovich SDE

dXdt

= h(X ) +

√2Dα

f (X ) dVdt.Pavliotis (IC) StochProc January 16, 2011 250 / 367

A Stratonovich SDE

dX (t) = f (X (t)) dt + σ(X (t)) dW (t) (121)

can be written as an Itô SDE

dX (t) =

(f (X (t)) +

12

(σ

dσdx

)(X (t))

)dt + σ(X (t)) dW (t).

Conversely, and Itô SDE

dX (t) = f (X (t)) dt + σ(X (t))dW (t) (122)

can be written as a Statonovich SDE

dX (t) =

(f (X (t)) − 1

2

(σ

dσdx

)(X (t))

)dt + σ(X (t)) dW (t).

The Itô and Stratonovich interpretation of an SDE can lead toequations with very different properties!


Multiplicative Noise.When the diffusion coefficient depends on the solution of the SDEX(t), we will say that we have an equation with multiplicativenoise .Multiplicative noise can lead to noise induced phase transitions .See

W. Horsthemke and R. Lefever, Noise-induced transitions,Springer-Verlag, Berlin 1984.

This is a topic of current interest for SDEs in infinite dimensions(SPDEs).


Colored Noise

When the noise which drives an SDE has non-zero correlationtime we will say that we have colored noise .The properties of the SDE (stability, ergodicity etc.) are quiterobust under "coloring of the noise". See

G. Blankenship and G.C. Papanicolaou, Stability and control ofstochastic systems with wide-band noise disturbances. I, SIAM J.Appl. Math., 34(3), 1978, pp. 437–476.

Colored noise appears in many applications in physics andchemistry. For a review see

P. Hanggi and P. Jung Colored noise in dynamical systems. Adv.Chem. Phys. 89 239 (1995).


In the case where there is an additional small time scale in theproblem, in addition to the correlation time of the colored noise, itis not clear what the right interpretation of the stochastic integral(in the limit as both small time scales go to 0). This is usuallycalled the Itô versus Stratonovich problem .

Consider, for example, the SDE

τ X = −X + v(X )ηε(t),

where ηε(t) is colored noise with correlation time ε2.In the limit where both small time scales go to 0 we can get eitherItô or Stratonovich or neither. See

G.A. Pavliotis and A.M. Stuart, Analysis of white noise limits forstochastic systems with two fast relaxation times, Multiscale Model.Simul., 4(1), 2005, pp. 1-35.


Given the function γ(z) in the SDE (105) we define

Γ(z) = γ(z)γ(z)T . (123)

The generator L is then defined as

Lv = h · ∇v +12

Γ : ∇∇v . (124)

This operator, equipped with a suitable domain of definition, is thegenerator of the Markov process given by (105).

The formal L2−adjoint operator L∗

L∗v = −∇ · (hv) +12∇ · ∇ · (Γv).


The Itô formula enables us to calculate the rate of change in timeof functions V : Z → R

n evaluated at the solution of a Z-valuedSDE.

Formally, we can write:

ddt

(V (z(t))

)= LV (z(t)) +

⟨∇V (z(t)), γ(z(t))

dWdt

⟩.

Note that if W were a smooth time-dependent function thisformula would not be correct: there is an additional term in LV ,proportional to Γ, which arises from the lack of smoothness ofBrownian motion.

The precise interpretation of the expression for the rate of changeof V is in integrated form:


Lemma(Itô’s Formula) Assume that the conditions of Theorem 60 hold. Letx(t) solve (105) and let V ∈ C2(Z,Rn). Then the process V (z(t))satisfies

V (z(t)) = V (z(0)) +

∫ t

0LV (z(s))ds +

∫ t

0〈∇V (z(s)), γ(z(s)) dW (s)〉 .

Let φ : Z 7→ R and consider the function

v(z, t) = E(φ(z(t))|z(0) = z

), (125)

where the expectation is with respect to all Brownian drivingpaths. By averaging in the Itô formula, which removes thestochastic integral, and using the Markov property, it is possible toobtain the Backward Kolmogorov equation.


For a Stratonovich SDE the rules of standard calculus apply:Consider the Stratonovich SDE (121) and let V (x) ∈ C2(R). Then

dV (X(t)) =dVdx

(X(t)) (f (X(t)) dt + σ(X(t)) dW (t)) .

Consider the Stratonovich SDE (121) on Rd (i.e. f ∈ Rd ,σ : Rn 7→ Rd , W (t) is standard Brownian motion on Rn). Thecorresponding Fokker-Planck equation is:

∂ρ

∂t= −∇ · (fρ) +

12∇ · (σ∇ · (σρ))). (126)


1 The SDE for Brownian motion is:

dX =√

2σdW , X (0) = x .

The Solution is:

X (t) = x +√

2σW (t).


2. The SDE for the Ornstein-Uhlenbeck process is

dX = −αX dt +√

2λdW , X (0) = x .

We can solve this equation using the variation of constants formula:

X(t) = e−αt x +√

2λ∫ t

0e−α(t−s)dW (s).

We can use Itô’s formula to obtain equations for the moments of theOU process. The generator is:

L = −αx∂x + λ∂2x .


We apply Itô’s formula to the function f (x) = xn to obtain:

dX (t)n = LX (t)n dt +√

2λ∂X (t)n dW

= −αnX (t)n dt + λn(n − 1)X (t)n−2 dt

+n√

2λX (t)n−1 dW .

Consequently:

X (t)n = xn +

∫ t

0

(−αnX (t)n + λn(n − 1)X (t)n−2

)dt

+n√

2λ∫ t

0X (t)n−1 dW .

By taking the expectation in the above equation we obtain theequation for the moments of the OU process that we derivedearlier using the Fokker-Planck equation:

Mn(t) = xn +

∫ t

0(−αnMn(s) + λn(n − 1)Mn−2(s)) ds.


3. Consider the geometric Brownian motion

dX (t) = µX (t) dt + σX (t) dW (t), (127)

where we use the Itô interpretation of the stochastic differential. Thegenerator of this process is

L = µx∂x +σ2x2

2∂2

x .

The solution to this equation is

X(t) = X(0) exp(

(µ− σ2

2)t + σW (t)

). (128)


To derive this formula, we apply Itô’s formula to the functionf (x) = log(x):

d log(X (t)) = L(

log(X (t)))

dt + σx∂x log(X (t)) dW (t)

=

(µx

1x

+σ2x2

2

(− 1

x2

))dt + σ dW (t)

=

(µ− σ2

2

)dt + σ dW (t).

Consequently:

log(

X (t)X (0)

)=

(µ− σ2

2

)t + σW (t)

from which (128) follows.


Notice that the Stratonovich interpretation of this equation leads tothe solution

X (t) = X (0) exp(µt + σW (t))

Exercise: calculate all moments of the geometric Brownian motionfor the Itô and Stratonovich interpretations of the stochasticintegral.


Consider the Landau equation :

dXt

dt= Xt(c − X 2

t ), X0 = x . (129)

This is a gradient flow for the potential V (x) = 12cx2 − 1

4x4.

When c < 0 all solutions are attracted to the single steady stateX∗ = 0.

When c > 0 the steady state X∗ = 0 becomes unstable andXt →

√c if x > 0 and Xt → −√

c if x < 0.


Consider additive random perturbations to the Landau equation:

dXt

dt= Xt(c − X 2

t ) +√

2σdWt

dt, X0 = x . (130)

This equation defines an ergodic Markov process on R: Thereexists a unique invariant distribution:

ρ(x) = Z−1e−V (x)/σ , Z =

∫

R

e−V (x)/σ dx , V (x) =12

cx2 − 14

x4.

ρ(x) is a probability density for all values of c ∈ R.

The presence of additive noise in some sense "trivializes" thedynamics.

The dependence of various averaged quantities on c resemblesthe physical situation of a second order phase transition.


Consider now multiplicative perturbations of the Landau equation.

dXt

dt= Xt(c − X 2

t ) +√

2σXtdWt

dt, X0 = x . (131)

Where the stochastic differential is interpreted in the Itô sense.

The generator of this process is

L = x(c − x2)∂x + σx2∂2x .

Notice that Xt = 0 is always a solution of (131). Thus, if we startwith x > 0 (x < 0) the solution will remain positive (negative).

We will assume that x > 0.


Consider the function Yt = log(Xt). We apply Itô’s formula to thisfunction:

dYt = L log(Xt) dt + σXt∂x log(Xt) dWt

=

(Xt(c − X 2

t )1Xt

− σX 2t

1X 2

t

)dt + σXt

1Xt

dWt

= (c − σ) dt − X 2t dt + σ dWt .

Thus, we have been able to transform (131) into an SDE withadditive noise:

dYt =[(c − σ) − e2Yt

]dt + σ dWt . (132)

This is a gradient flow with potential

V (y) = −[(c − σ)y − 1

2e2y].


The invariant measure, if it exists, is of the form

ρ(y) dy = Z−1e−V (y)/σ dy .

Going back to the variable x we obtain:

ρ(x) dx = Z−1x (c/σ−2)e− x2

2σ dx .

We need to make sure that this distribution is integrable:

Z =

∫ +∞

0xγe− x2

2σ <∞, γ =cσ− 2.

For this it is necessary that

γ > −1 ⇒ c > σ.


Not all multiplicative random perturbations lead to ergodicbehavior!

The dependence of the invariant distribution on c is similar to thephysical situation of first order phase transitions.

Exercise Analyze this problem for the Stratonovich interpretationof the stochastic integral.

Exercise Study additive and multiplicative random perturbationsof the ODE

dxdt

= x(c + 2x2 − x4).

For more information see M.C. Mackey, A. Longtin, A. LasotaNoise-Induced Global Asymptotic Stability , J. Stat. Phys. 60(5/6) pp. 735-751.


Theorem

Assume that φ is chosen sufficiently smooth so that the backwardKolmogorov equation

∂v∂t

= Lv for (z, t) ∈ Z × (0,∞),

v = φ for (z, t) ∈ Z × 0 , (133)

has a unique classical solution v(x , t) ∈ C2,1(Z × (0,∞), ). Then v isgiven by (125) where z(t) solves (106).


Now we can derive rigorously the Fokker-Planck equation.

Theorem

Consider equation (106) with z(0) a random variable with densityρ0(z). Assume that the law of z(t) has a densityρ(z, t) ∈ C2,1(Z × (0,∞)). Then ρ satisfies the Fokker-Planckequation

∂ρ

∂t= L∗ρ for (z, t) ∈ Z × (0,∞), (134a)

ρ = ρ0 for z ∈ Z × 0. (134b)


Proof

Let Eµ denote averaging with respect to the product measure

induced by the measure µ with density ρ0 on z(0) and theindependent driving Wiener measure on the SDE itself.

Averaging over random z(0) distributed with density ρ0(z), we find

Eµ(φ(z(t))) =

∫

Zv(z, t)ρ0(z) dz

=

∫

Z(eLtφ)(z)ρ0(z) dz

=

∫

Z(eL∗tρ0)(z)φ(z) dz.

But since ρ(z, t) is the density of z(t) we also have

Eµ(φ(z(t))) =

∫

Zρ(z, t)φ(z)dz.


Equating these two expressions for the expectation at time t weobtain ∫

Z(eL∗tρ0)(z)φ(z) dz =

∫

Zρ(z, t)φ(z) dz.

We use a density argument so that the identity can be extended toall φ ∈ L2(Z). Hence, from the above equation we deduce that

ρ(z, t) =(

eL∗tρ0

)(z).

Differentiation of the above equation gives (134a).

Setting t = 0 gives the initial condition (134b).


THE SMOLUCHOWSKI AND FREIDLIN-WENTZELL LIMITS


There are very few SDEs/Fokker-Planck equations that can besolved explicitly.

In most cases we need to study the problem under investigationeither approximately or numerically.

In this part of the course we will develop approximate methods forstudying various stochastic systems of practical interest.


There are many problems of physical interest that can beanalyzed using techniques from perturbation theory andasymptotic analysis:

1 Small noise asymptotics at finite time intervals.2 Small noise asymptotics/large times (rare events): the theory of

large deviations, escape from a potential well, exit time problems.3 Small and large friction asymptotics for the Fokker-Planck

equation: The Freidlin–Wentzell (underdamped) andSmoluchowski (overdamped) limits.

4 Large time asymptotics for the Langevin equation in a periodicpotential: homogenization and averaging.

5 Stochastic systems with two characteristic time scales: multiscaleproblems and methods.


We will study various asymptotic limits for the Langevin equation(we have set m = 1)

q = −∇V (q) − γq +√

2γβ−1W . (135)

There are two parameters in the problem, the friction coefficient γand the inverse temperature β.

We want to study the qualitative behavior of solutions to thisequation (and to the corresponding Fokker-Planck equation).

There are various asymptotic limits at which we can eliminatesome of the variables of the equation and obtain a simplerequation for fewer variables.

In the large temperature limit, β ≪ 1, the dynamics of (183) isdominated by diffusion: the Langevin equation (183) can beapproximated by free Brownian motion:

q =√

2γβ−1W .


The small temperature asymptotics, β ≫ 1 is much moreinteresting and more subtle. It leads to exponential, Arrheniustype asymptotics for the reaction rate (in the case of a particleescaping from a potential well due to thermal noise) or thediffusion coefficient (in the case of a particle moving in a periodicpotential in the presence of thermal noise)

κ = ν exp (−βEb) , (136)

where κ can be either the reaction rate or the diffusion coefficient.The small temperature asymptotics will be studied later for thecase of a bistable potential (reaction rate) and for the case of aperiodic potential (diffusion coefficient).


Assuming that the temperature is fixed, the only parameter that isleft is the friction coefficient γ. The large and small frictionasymptotics can be expressed in terms of a slow/fast system ofSDEs.

In many applications (especially in biology) the friction coefficientis large: γ ≫ 1. In this case the momentum is the fast variablewhich we can eliminate to obtain an equation for the position. Thisis the overdamped or Smoluchowski limit.

In various problems in physics the friction coefficient is small:γ ≪ 1. In this case the position is the fast variable whereas theenergy is the slow variable. We can eliminate the position andobtain an equation for the energy. This is the underdampled orFreidlin-Wentzell limit.

In both cases we have to look at sufficiently long time scales.


We rescale the solution to (183):

qγ(t) = λγq(t/µγ).

This rescaled process satisfies the equation

qγ = −λγµ2γ

∂qV (qγ/λγ) −γ

µγqγ +

√2γλ2

γµ−3γ β−1W , (137)

Different choices for these two parameters lead to theoverdamped and underdamped limits:


λγ = 1, µγ = γ−1, γ ≫ 1.

In this case equation (137) becomes

γ−2qγ = −∂qV (qγ) − qγ +√

2β−1W . (138)

Under this scaling, the interesting limit is the overdamped limit,γ ≫ 1.

We will see later that in the limit as γ → +∞ the solution to (138)can be approximated by the solution to

q = −∂qV +√

2β−1W .


λγ = 1, µγ = γ, γ ≪ 1:

qγ = −γ−2∇V (qγ) − qγ +√

2γ−2β−1W . (139)

Under this scaling the interesting limit is the underdamped limit,γ ≪ 1.

We will see later that in the limit as γ → 0 the energy of thesolution to (139) converges to a stochastic process on a graph.


We consider the rescaled Langevin equation (138):

ε2qγ(t) = −∇V (qγ(t)) − qγ(t) +√

2β−1W (t), (140)

where we have set ε−1 = γ, since we are interested in the limitγ → ∞, i.e. ε→ 0.

We will show that, in the limit as ε→ 0, qγ(t), the solution of theLangevin equation (140), converges to q(t), the solution of theSmoluchowski equation

q = −∇V +√

2β−1W . (141)


We write (140) as a system of SDEs:

q =1ε

p, (142)

p = −1ε∇V (q) − 1

ε2 p +

√2βε2 W . (143)

This systems of SDEs defined a Markov process in phase space.Its generator is

Lε =1ε2

(− p · ∇p + β−1∆p

)+

1ε

(p · ∇q −∇qV · ∇p

)

=:1ε2L0 +

1εL1.

This is a singularly perturbed differential operator.

We will derive the Smoluchowski equation (141) using a pathwisetechnique, as well as by analyzing the corresponding Kolmogorovequations.


We apply Itô’s formula to p:

dp(t) = Lεp(t) dt +1ε

√2β−1∂pp(t) dW

= − 1ε2 p(t) dt − 1

ε∇qV (q(t)) dt +

1ε

√2β−1 dW .

Consequently:

1ε

∫ t

0p(s) ds = −

∫ t

0∇qV (q(s)) ds +

√2β−1 W (t) + O(ε).

From equation (142) we have that

q(t) = q(0) +1ε

∫ t

0p(s) ds.

Combining the above two equations we deduce

q(t) = q(0) −∫ t

0∇qV (q(s)) ds +

√2β−1 W (t) + O(ε)

from which (141) follows.


Notice that in this derivation we assumed that

E|p(t)|2 6 C.

This estimate is true, under appropriate assumptions on thepotential V (q) and on the initial conditions.

In fact, we can prove a pathwise approximation result:

(E sup

t∈[0,T ]

|qγ(t) − q(t)|p)1/p

6 Cε2−κ,

where κ > 0, arbitrary small (it accounts for logarithmiccorrections).


For the rigorous proof seeE. Nelson, Dynamical Theories of Brownian Motion, PrincetonUniversity press 1967.

A similar approximation theorem is also valid in infinite dimensions(i.e. for SPDEs):

S. Cerrai and M. Freidlin, On the Smoluchowski-Kramersapproximation for a system with an infinite number of degrees offreedom, Probab. Theory Related Fields, 135 (3), 2006, pp.363–394.


The pathwise derivation of the Smoluchowski equation impliesthat the solution of the Fokker-Planck equation corresponding tothe Langevin equation (140) converges (in some appropriatesense to be explained below) to the solution of the Fokker-Planckequation corresponding to the Smoluchowski equation (141).

It is important in various applications to calculate corrections tothe limiting Fokker-Planck equation.

We can accomplish this by analyzing the Fokker-Planck equationfor (140) using singular perturbation theory.

We will consider the problem in one dimension. This mainly tosimplify the notation. The multi–dimensional problem can betreated in a very similar way.


The Fokker–Planck equation associated to equations (142) and(143) is

∂ρ

∂t= L∗ρ

=1ε

(−p∂qρ+ ∂qV (q)∂pρ) +1ε2

(∂p(pρ) + β−1∂2

pρ)

=:

(1ε2L

∗0 +

1εL∗

1

)ρ. (144)

The invariant distribution of the Markov process q, p, if it exists,is

ρ0(p,q) =1Z

e−βH(p,q), Z =

∫

R2e−βH(p,q) dpdq,

where H(p,q) = 12p2 + V (q). We define the function f(p,q,t)

throughρ(p,q, t) = f (p,q, t)ρ0(p,q). (145)


TheoremThe function f (p,q, t) defined in (145) satisfies the equation

∂f∂t

=

[1ε2

(−p∂q + β−1∂2

p

)− 1ε

(p∂q − ∂qV (q)∂p)

]f

=:

(1ε2L0 −

1εL1

)f . (146)

remark

This is "almost" the backward Kolmogorov equation with thedifference that we have −L1 instead of L1. This is related to thefact that L0 is a symmetric operator in L2(R2; Z−1e−βH(p,q)),whereas L1 is antisymmetric.


Proof.We note that L∗

0ρ0 = 0 and L∗1ρ0 = 0. We use this to calculate:

L∗0ρ = L0(fρ0) = ∂p(fρ0) + β−1∂2

p(fρ0)

= ρ0p∂pf + ρ0β−1∂2

p f + fL∗0ρ0 + 2β−1∂pf∂pρ0

=(−p∂pf + β−1∂2

p f)ρ0 = ρ0L0f .

Similarly,

L∗1ρ = L∗

1(fρ0) = (−p∂q + ∂qV∂p) (fρ0)

= ρ0 (−p∂qf + ∂qV∂pf ) = −ρ0L1f .

Consequently, the Fokker–Planck equation (144) becomes

ρ0∂f∂t

= ρ0

(1ε2L0f − 1

εL1f),

from which the claim follows.


We look for a solution to (146) in the form of a power series in ε:

f (p,q, t) =∞∑

n=0

εnfn(p,q, t). (147)

We substitute this expansion into eqn. (146) to obtain the followingsystem of equations.

L0f0 = 0, (148)

L0f1 = L1f0, (149)

L0f2 = L1f1 +∂f0∂t

(150)

L0fn+1 = L1fn +∂fn∂t, n = 2,3 . . . (151)

The null space of L0 consists of constants in p. Consequently,from equation (148) we conclude that

f0 = f (q, t).


Now we can calculate the right hand side of equation (149):

L1f0 = p∂qf .

Equation (149) becomes:

L0f1 = p∂qf .

The right hand side of this equation is orthogonal to N (L∗0) and

consequently there exists a unique solution. We obtain thissolution using separation of variables:

f1 = −p∂qf + ψ1(q, t).


Now we can calculate the RHS of equation (150). We need tocalculate L1f1:

−L1f1 =(

p∂q − ∂qV∂p

)(p∂qf − ψ1(q, t)

)

= p2∂2q f − p∂qψ1 − ∂qV∂q f .

The solvability condition for (150) is∫

R

(− L1f1 −

∂f0∂t

)ρOU(p) dp = 0,

from which we obtain the backward Kolmogorov equationcorresponding to the Smoluchowski SDE:

∂f∂t

= −∂qV∂qf + β−1∂2p f . (152)

Now we solve the equation for f2. We use (152) to write (150) inthe form

L0f2 =(β−1 − p2

)∂2

q f + p∂qψ1.



f2(p,q, t) =12∂2

q f (p,q, t)p2 − ∂qψ1(q, t)p + ψ2(q, t).

The right hand side of the equation for f3 is

L1f2 =12

p3∂3q f − p2∂2

qψ1 + p∂qψ2 − ∂qV∂2q fp − ∂qV∂qψ1.

The solvability condition∫

R

L1f2ρOU(p) dp = 0.

This leads to the equation

−∂qV∂qψ1 − β−1∂2qψ1 = 0

The only solution to this equation which is an element ofL2(e−βV (q)) is

ψ1 ≡ 0.


Putting everything together we obtain the first two terms in theε-expansion of the Fokker–Planck equation (146):

ρ(p,q, t) = Z−1e−βH(p,q)(

f + ε(−p∂qf ) + O(ε2)),

where f is the solution of (152).

Notice that we can rewrite the leading order term to the expansionin the form

ρ(p,q, t) = (2πβ−1)−12 e−βp2/2ρV (q, t) + O(ε),

where ρV = Z−1e−βV (q)f is the solution of the SmoluchowskiFokker-Planck equation

∂ρV

∂t= ∂q(∂qVρV ) + β−1∂2

qρV .


It is possible to expand the n-th term in the expansion (147) interms of Hermite functions (the eigenfunctions of the generator ofthe OU process)

fn(p,q, t) =

n∑

k=0

fnk (q, t)φk (p), (153)

where φk (p) is the k–th eigenfunction of L0:

−L0φk = λkφk .

We can obtain the following system of equations(L = β−1∂q − ∂qV ):

Lfn1 = 0,√k + 1β−1 Lfn,k+1 +

√kβ−1∂q fn,k−1 = −kfn+1,k , k = 1,2 . . . ,n − 1,

√nβ−1∂qfn,n−1 = −nfn+1,n,√

(n + 1)β−1∂qfn,n = −(n + 1)fn+1,n+1.


Using this method we can obtain the first three terms in theexpansion:

ρ(x , y , t) = ρ0(p,q)(

f + ε(−√β−1∂q fφ1)

+ε2(β−1√

2∂2

q fφ2 + f20)

+ε3(−√β−3

3!∂3

q fφ3 +(−√β−1L∂2

q f

−√β−1∂q f20

)φ1

))

+O(ε4),


The Freilin–Wentzell Limit

Consider now the rescaling λγ,ε = 1, µγ,ε = γ. The Langevin equationbecomes

qγ = −γ−2∇V (qγ) − qγ +√

2γ−2β−1W . (154)

We write equation (154) as system of two equations

qγ = γ−1pγ , pγ = −γ−1V ′(qγ) − pγ +√

2β−1W .

This is the equation for an O(1/γ) Hamiltonian system perturbed byO(1) noise. We expect that, to leading order, the energy is conserved,since it is conserved for the Hamiltonian system. We apply Itô’sformula to the Hamiltonian of the system to obtain

H =(β−1 − p2

)+

√2β−1p2W

with p2 = p2(H,q) = 2(H − V (q)).


Thus, in order to study the γ → 0 limit we need to analyze the followingfast/slow system of SDEs

H =(β−1 − p2

)+√

2β−1p2W (155a)

pγ = −γ−1V ′(qγ) − pγ +√

2β−1W . (155b)

The Hamiltonian is the slow variable, whereas the momentum (orposition) is the fast variable. Assuming that we can average over theHamiltonian dynamics, we obtain the limiting SDE for the Hamiltonian:

H =(β−1 − 〈p2〉

)+√

2β−1〈p2〉W . (156)

The limiting SDE lives on the graph associated with the Hamiltoniansystem. The domain of definition of the limiting Markov process isdefined through appropriate boundary conditions (the gluingconditions ) at the interior vertices of the graph.


We identify all points belonging to the same connectedcomponent of the a level curve x : H(x) = H, x = (q,p).

Each point on the edges of the graph correspond to a trajectory.

Interior vertices correspond to separatrices.

Let Ii , i = 1, . . . d be the edges of the graph. Then (i ,H) defines aglobal coordinate system on the graph.For more information see

Freidlin and Wentzell, Random Perturbations of DynamicalSystems, Springer 1998.Freidlin and Wentzell, Random Perturbations of HamiltonianSystems, AMS 1994.Sowers, A Boundary layer theory for diffusively perturbed transportaround a heteroclinic cycle, CPAM 58 (2005), no. 1, 30–84.


We will study the small γ asymptotics by analyzing the correspondingbackward Kolmogorov equation using singular perturbation theory.The generator of the process qγ , pγ is

Lγ = γ−1 (p∂q − ∂qV∂p) − p∂p + β−1∂2p

= γ−1L0 + L1.

Let uγ = E(f (pγ(p,q; t),qγ(p,q; t))). It satisfies the backwardKolmogorov equation associated to the process qγ , pγ:

∂uγ

∂t=

(1γL0 + L1

)uγ . (157)


We look for a solution in the form of a power series expansion in ε:

uγ = u0 + γu1 + γ2u2 + . . .

We substitute this ansatz into (157) and equate equal powers in ε toobtain the following sequence of equations:

L0u0 = 0, (158a)

L0u1 = −L1u1 +∂u0

∂t, (158b)

L0u2 = −L1u1 +∂u1

∂t. (158c)

. . . . . . . . .

Notice that the operator L0 is the backward Liouville operator of theHamiltonian system with Hamiltonian

H =12

p2 + V (q).


We assume that there are no integrals of motion other than theHamiltonian. This means that the null space of L0 consists of functionsof the Hamiltonian:

N (L0) =

functions ofH. (159)

Let us now analyze equations (158). We start with (158a); eqn. (159)implies that u0 depends on q, p through the Hamiltonian function H:

u0 = u(H(p,q), t) (160)

Now we proceed with (158b). For this we need to find the solvabilitycondition for equations of the form

L0u = f (161)

My multiply it by an arbitrary smooth function of H(p,q), integrate overR

2 and use the skew-symmetry of the Liouville operator L0 to deduce:11We assume that both u1 and F decay to 0 as |p| → ∞ to justify the integration by

parts that follows.Pavliotis (IC) StochProc January 16, 2011 305 / 367

∫

R2L0uF (H(p,q)) dpdq =

∫

R2uL∗

0F (H(p,q)) dpdq

=

∫

R2u(−L0F (H(p,q))) dpdq

= 0, ∀F ∈ C∞b (R).

This implies that the solvability condition for equation (161) is that∫

R2f (p,q)F (H(p,q)) dpdq = 0, ∀F ∈ C∞

b (R). (162)

We use the solvability condition in (158b) to obtain that∫

R2

(L1u1 −

∂u0

∂t

)F (H(p,q)) dpdq = 0, (163)

To proceed, we need to understand how L1 acts to functions ofH(p,q). Let φ = φ(H(p,q)). We have that

∂φ

∂p=∂H∂p

∂φ

∂H= p

∂φ

∂H

andPavliotis (IC) StochProc January 16, 2011 306 / 367

∂2φ

∂p2 =∂

∂p

(∂φ

∂H

)=∂φ

∂H+ p2 ∂

2φ

∂H2 .

The above calculations imply that, when L1 acts on functionsφ = φ(H(p,q)), it becomes

L1 =[(β−1 − p2)∂H + β−1p2∂2

H

], (164)

wherep2 = p2(H,q) = 2(H − V (q)).


We want to change variables in the integral (163) and go from (p, q) top, H. The Jacobian of the transformation is:

∂(p,q)

∂(H,q)=

∂p∂H

∂p∂q

∂q∂H

∂q∂q

=∂p∂H

=1

p(H,q).

We use this, together with (164), to rewrite eqn. (163) as∫ ∫ (∂u

∂t+[(β−1 − p2)∂H + β−1p2∂2

H

]u)

F (H)p−1(H,q) dHdq

= 0.

We introduce the notation

〈·〉 :=

∫·dq.

The integration over q can be performed "explicitly":∫ [∂u

∂t〈p−1〉 +

((β−1〈p−1〉 − 〈p〉)∂H + β−1〈p〉∂2

H

)u]F (H) dH = 0.


This equation should be valid for every smooth function F (H), and thisrequirement leads to the differential equation

〈p−1〉∂u∂t

=(β−1〈p−1〉 − 〈p〉

)∂Hu + 〈p〉β−1∂2

Hu,

or,∂u∂t

=(β−1 − 〈p−1〉−1〈p〉

)∂Hu + γ〈p−1〉−1〈p〉β−1∂2

Hu.

Thus, we have obtained the limiting backward Kolmogorov equation forthe energy, which is the "slow variable". From this equation we canread off the limiting SDE for the Hamiltonian:

H = b(H) + σ(H)W (165)

where

b(H) = β−1 − 〈p−1〉−1〈p〉, σ(H) = β−1〈p−1〉−1〈p〉.


Notice that the noise that appears in the limiting equation (165) ismultiplicative, contrary to the additive noise in the Langevin equation.As it well known from classical mechanics, the action and frequencyare defined as

I(E) =

∫p(q,E) dq

and

ω(E) = 2π(

dIdE

)−1

,

respectively. Using the action and the frequency we can write thelimiting Fokker–Planck equation for the distribution function of theenergy in a very compact form.

TheoremThe limiting Fokker–Planck equation for the energy distributionfunction ρ(E , t) is

∂ρ

∂t=

∂

∂E

((I(E) + β−1 ∂

∂E

)(ω(E)ρ

2π

)). (166)


Proof.We notice that

dIdE

=

∫∂p∂E

dq =

∫p−1 dq

and consequently 〈p−1〉−1 = ω(E)2π . Hence, the limiting Fokker–Planck

equation can be written as

∂ρ

∂t= − ∂

∂E

((β−1 I(E)ω(E)

2π

)ρ

)+ β−1 ∂2

∂E2

(Iω2π

)

= −β−1 ∂ρ

∂E+

∂

∂E

(Iω2πρ

)+ β−1 ∂

∂E

(dIdE

ωρ

2π

)

+β−1 ∂

∂E

(I∂

∂E

(ωρ2π

))

=∂

∂E

(Iω2πρ

)+ β−1 ∂

∂E

(I∂

∂E

(ωρ2π

))

=∂

∂E

((I(E) + β−1 ∂

∂E

)(ω(E)ρ

2π

)),

which is precisely equation (166).Pavliotis (IC) StochProc January 16, 2011 311 / 367

Remarks

1 We emphasize that the above formal procedure does not provideus with the boundary conditions for the limiting Fokker–Planckequation. We will discuss about this issue in the next section.

2 If we rescale back to the original time-scale we obtain the equation

∂ρ

∂t= γ

∂

∂E

((I(E) + β−1 ∂

∂E

)(ω(E)ρ

2π

)). (167)

We will use this equation later on to calculate the rate of escapefrom a potential barrier in the energy-diffusion-limited regime.


Exit Time Problems and Escape from a Potential Well


Escape From a Potential Well

There are many systems in physics, chemistry and biology thatexist in at least two stable states:

Switching and storage devices in computers.Conformational changes of biological macromolecules (they canexist in many different states).

The problems that we would like to study are:How stable are the various states relative to each other.How long does it take for a system to switch spontaneously fromone state to another?How is the transfer made, i.e. through what path in the relevantstate space?How does the system relax to an unstable state?


The study of bistability and metastability is a very active researcharea. Topics of current research include:

The development of numerical methods for the calculation ofvarious quantities such as reaction rates, transition pathways etc.The study of bistable systems in infinite dimensions.


We will consider the dynamics of a particle moving in a bistablepotential, under the influence of thermal noise in one dimension:

x = −V ′(x) +√

2kBT β. (168)

We will consider potentials that have to local minima, one localmaximum (saddle point) and they increase at least quadraticallyfast at infinity. This ensures that the state space is "compact", i.e.that the particle cannot escape at infinity.

The standard potential that satisfies these assumptions is

V (x) =14

x4 − 12

x2 +14. (169)

This potential has three local minima, a local maximum at x = 0and two local minima at x = ±1.


0 50 100 150 200 250 300 350 400 450 500−3

−2

−1

0

1

2

3

Figure: Sample Path of solution to equation (168).


The values of the potential at these three points are:

V (±1) = 0, V (0) =14.

We will say that the height of the potential barrier is 14 . The

physically (and mathematically!) interesting case is when thethermal fluctuations are weak when compared to the potentialbarrier that the particle has to climb over.

More generally, we assume that the potential has two local minimaat the points a and c and a local maximum at b.

We will consider the problem of the escape of the particle from theleft local minimum a.

The potential barrier is then defined as

Eb = V (b)− V (a).


Our assumption that the thermal fluctuations are weak can bewritten as

kBTEb

≪ 1. (170)

Under condition (170), the particle is most likely to be found closeto a or c. There it will perform small oscillations around either ofthe local minima.

We can study the weak noise asymptotics for finite times usingstandard perturbation theory.

We can describe locally the dynamics of the particle byappropriate Ornstein-Uhlenbeck processes.

This result is valid only for finite times: at sufficiently long timesthe particle can escape from the one local minimum, a say, andsurmount the potential barrier to end up at c. It will then spend along time in the neighborhood of c until it escapes again thepotential barrier and end at a.


This is an example of a rare event . The relevant time scale, theexit time or the mean first passage time scales exponentially inβ := (kBT )−1:

τ = ν−1 exp(βEb).

the rate with which particles escape from a local minimum of thepotential is called the rate of escape or the reaction rateκ := τ−1:

κ = ν exp(−β∆E). (171)

It is important to notice that the escape from a local minimum, i.e.a state of local stability, can happen only at positive temperatures:it is a noise assisted event :

In the absence of thermal fluctuations the equation of motionbecomes:

x = −V ′(x), x(0) = x0.


In this case the potential becomes a Lyapunov function :

dxdt

= V ′(x)dxdt

= −(V ′(x))2 < 0.

Hence, depending on the initial condition the particle will convergeeither to a or c. The particle cannot escape from either state oflocal stability.

On the other hand, at high temperatures the particle does not"see" the potential barrier: it essentially jumps freely from onelocal minimum to another.

We will study this problem and calculate the escape rate using bycalculating the mean first passage time .


The Arrhenius-type factor in the formula for the reaction rate,eqn. (171) is intuitively clear and it has been observedexperimentally in the late nineteenth century by Arrhenius andothers.

What is extremely important both from a theoretical and anapplied point of view is the calculation of the prefactor ν, the ratecoefficient .

A systematic approach for the calculation of the rate coefficient, aswell as the justification of the Arrhenius kinetics, is that of themean first passage time method (MFPT).

Since this method is of independent interest and is useful invarious other contexts, we will present it in a quite general settingand apply it to the problem of the escape from a potential barrierlater on.


Let Xt be a continuous time diffusion process on Rd whose

evolution is governed by the SDE

dX xt = b(X x

t ) dt + σ(X xt ) dWt , X x

0 = x . (172)

Let D be a bounded subset of Rd with smooth boundary. Given

x ∈ D, we want to know how long it takes for the process Xt toleave the domain D for the first time

τ xD = inf t > 0 : X x

t /∈ D .

Clearly, this is a random variable. The average of this randomvariable is called the mean first passage time MFPT or the firstexit time :

τ(x) := Eτ xD.

We can calculate the MFPT by solving an appropriate boundaryvalue problem.


Theorem

The MFPT is the solution of the boundary value problem

−Lτ = 1, x ∈ D, (173a)

τ = 0, x ∈ ∂D, (173b)

where L is the generator of the SDE 173.

The homogeneous Dirichlet boundary conditions correspond to anabsorbing boundary : the particles are removed when they reachthe boundary. Other choices of boundary conditions are alsopossible.

The rigorous proof of Theorem 67 is based on Itô’s formula.


Derivation of the BVP for the MFPT

Let ρ(X , x , t) be the probability distribution of the particles thathave not left the domain D at time t . It solves the FP equationwith absorbing boundary conditions.

∂ρ

∂t= L∗ρ, ρ(X , x ,0) = δ(X − x), ρ|∂D = 0. (174)

We can write the solution to this equation in the form

ρ(X , x , t) = eL∗tδ(X − x),

where the absorbing boundary conditions are included in thedefinition of the semigroup eL∗t .The homogeneous Dirichlet (absorbing) boundary conditionsimply that

limt→+∞

ρ(X , x , t) = 0.

That is: all particles will eventually leave the domain.


The (normalized) number of particles that are still inside D at timet is

S(x , t) =

∫

Dρ(X , x , t) dx .

Notice that this is a decreasing function of time. We can write

∂S∂t

= −f (x , t),

where f (x , t) is the first passage times distribution .


The MFPT is the first moment of the distribution f (x , t):

τ(x) =

∫ +∞

0f (s, x)s ds =

∫ +∞

0−dS

dss ds

=

∫ +∞

0S(s, x) ds =

∫ +∞

0

∫

Dρ(X , x , s) dXds

=

∫ +∞

0

∫

DeL∗sδ(X − x) dXds

=

∫ +∞

0

∫

Dδ(X − x)

(eLs1

)dXds =

∫ +∞

0

(eLs1

)ds.

We apply L to the above equation to deduce:

Lτ =

∫ +∞

0

(LeLt1

)dt =

∫ t

0

ddt

(eLt1

)dt

= −1.


Consider the boundary value problem for the MFPT of the onedimensional diffusion process (168) from the interval (a,b):

− β−1eβV∂x

(e−βV∂xτ

)= 1 (175)

We choose reflecting BC at x = a and absorbing B.C. at x = b.We can solve (175) with these boundary conditions byquadratures:

τ(x) = β−1∫ b

xdyeβV (y)

∫ y

adze−βV (z). (176)

Now we can solve the problem of the escape from a potential well:the reflecting boundary is at x = a, the left local minimum of thepotential, and the absorbing boundary is at x = b, the localmaximum.


We can replace the B.C. at x = a by a repelling B.C. at x = −∞:

τ(x) = β−1∫ b

xdyeβV (y)

∫ y

−∞dze−βV (z).

When Ebβ ≫ 1 the integral wrt z is dominated by the value of thepotential near a. Furthermore, we can replace the upper limit ofintegration by ∞:

∫ z

−∞exp(−βV (z)) dz ≈

∫ +∞

−∞exp(−βV (a))

exp

(−βω

20

2(z − a)2

)dz

= exp (−βV (a))

√2πβω2

0

,

where we have used the Taylor series expansion around theminimum:

V (z) = V (a) +12ω2

0(z − a)2 + . . .


Similarly, the integral wrt y is dominated by the value of thepotential around the saddle point. We use the Taylor seriesexpansion

V (y) = V (b) − 12ω2

b(y − b)2 + . . .

Assuming that x is close to a, the minimum of the potential, wecan replace the lower limit of integration by −∞. We finally obtain

∫ b

xexp(βV (y)) dy ≈

∫ b

−∞exp(βV (b)) exp

(−βω

2b

2(y − b)2

)dy

=12

exp (βV (b))

√2πβω2

b

.


Putting everything together we obtain a formula for the MFPT:

τ(x) =π

ω0ωbexp (βEb) .

The rate of arrival at b is 1/τ . Only half of the particles escape.Consequently, the escape rate (or reaction rate), is given by 1

2τ :

κ =ω0ωb

2πexp (−βEb) . (177)


Consider now the problem of escape from a potential well for theLangevin equation

q = −∂qV (q) − γq +√

2γβ−1W . (178)

The reaction rate depends on the fiction coefficient and thetemperature. In the overdamped limit (γ ≫ 1) we retrieve (177),appropriately rescaled with γ:

κ =ω0ωb

2πγexp (−βEb) . (179)

We can also obtain a formula for the reaction rate for γ = O(1):

κ =

√γ2

4 − ω2b − γ

2

ωb

ω0

2πexp (−βEb) . (180)

Naturally, in the limit as γ → +∞ (180) reduces to (179)


In order to calculate the reaction rate in the underdamped orenergy-diffusion-limited regime γ ≪ 1 we need to study thediffusion process for the energy, (165) or (166). The result is

κ = γβI(Eb)ω0

2πe−βEb , (181)

where I(Eb) denotes the action evaluated at b.

A formula for the escape rate which is valid for all values of frictioncoefficient was obtained by Melnikov and Meshkov in 1986, J.Chem. Phys 85(2) 1018-1027. This formula requires thecalculation of integrals and it reduced to (179) and (181) in theoverdamped and underdamped limits, respectively.


The escape rate can be calculated in closed form only in 1d or inmultidimesional problems that can be reduced to a 1d problem(e.g., problems with radial symmetry).For more information on the calculation of escape rates andapplications of reaction rate theory see

P. Hänggi, P. Talkner and M. Borkovec: Reaction rate theory: fiftyyears after Kramers. Rev. Mod. Phys. 62(2) 251-341 (1990).R. Zwanzig: Nonequilibrium Statistical Mechanics, Oxford 2001,Ch. 4.C.W. Gardiner: Handbook of Stochastic Methods, Springer 2002,Ch. 9.Freidlin and Wentzell: Random Perturbations of DynamicalSystems, Springer 1998, Ch. 6.


The Kac-Zwanzig model and the Generalized Langevin Equatio n


Contents

Introduction: Einstein’s theory of Brownian motion.

Particle coupled to a heat bath: the Kac-Zwanzig model.

Coupled particle/field models.

The Mori-Zwanzig formalism.


Heavy (Brownian) particle immersed in a fluid.

Collisions of the heavy particle with the light fluid particles.

Statistical description of the motion of the Brownian particle.

Fluid is assumed to be in equilibrium.

Brownian particle performs an effective random walk:

〈X (t)2〉 = 2Dt .

The constant D is the diffusion coefficient (which should becalculated from first principles, i.e. through Green-Kubo formulas).


Phenomenological theories based either on an equation for theevolution of the probability distribution function ρ(q,p, t) (theFokker-Planck equation )

∂tρ = −p∂qρ+ ∂qV∂pρ+ γ(∂p(pρ) + β−1∂2

pρ)

(182)

or the a stochastic equation for the evolution of trajectories, theLangevin equation

q = −V ′(q) − γq +√

2γβ−1ξ, (183)

where ξ(t) is a white noise process, i.e. a (generalized) mean zeroGaussian Markov process with correlation function

〈ξ(t)ξ(s)〉 = δ(t − s).


Notice that the noise and dissipation in the Langevin equation arenot independent. They are related through thefluctuation-dissipation theorem .

This is not a coincidence since, as we will see later, noise anddissipation have the same source, namely the interaction betweenthe Brownian particle and its environment (heat bath).

There are two phenomenological constants, the inversetemperature β and the friction coefficient γ.

Our goal is to derive the Fokker-Planck and Langevin equationsfrom first principles and to calculate the phenomenologicalcoefficients β and γ.

We will consider some simple "particle + environment" systems forwhich we can obtain rigorously a stochastic equation thatdescribes the dynamics of the Brownian particle.


We can describe the dynamics of the Brownian particle/fluidsystem:

H(QN ,PN ; q,p) = HBP(QN ,PN) + HHB(q,p) + HI(QN ,q), (184)

where q, p :=qjN

j=1, pjNj=1

are the positions and

momenta of the fluid particles, N is the number of fluid (heat bath )particles (we will need to take the thermodynamic limit N → +∞).

The initial conditions of the Brownian particle are taken to be fixed,whereas the fluid is assumed to be initially in equilibrium (Gibbsdistribution).


Goal: eliminate the fluid variables q, p :=qjN

j=1, pjNj=1

to

obtain a closed equation for the Brownian particle.

We will see that this equation is a stochastic integrodifferentialequation, the Generalized Langevin Equation (GLE) (in the limitas N → +∞)

Q = −V ′(Q) −∫ t

0R(t − s)Q(s) ds + F (t), (185)

where R(t) is the memory kernel and F (t) is the noise .

We will also see that, in some appropriate limit, we can derive theMarkovian Langevin equation (183).


Need to model the interaction between the heat bath particles andthe coupling between the Brownian particle and the heat bath.

The simplest model is that of a harmonic heat bath and of linearcoupling:

H(QN ,PN ,q,p) =P2

N

2+ V (QN) +

N∑

n=1

p2n

2mn+

12

kn(qn − λQN)2.(186)

The initial conditions of the Brownian particleQN(0), PN(0) := Q0, P0 are taken to be deterministic.

The initial conditions of the heat bath particles are distributedaccording to the Gibbs distribution, conditional on the knowledgeof Q0, P0:

µβ(dpdq) = Z−1e−βH(q,p) dqdp, (187)

where β is the inverse temperature. This is a way of introducingthe concept of the temperature in the system (through the averagekinetic energy of the bath particles).


In order to choose the initial conditions according to µβ(dpdq) wecan take

qn(0) = λQ0 +

√β−1k−1

n ξn, pn(0) =√

mnβ−1ηn, (188)

where the ξn ηn are mutually independent sequences of i.i.d.N (0,1) random variables.

Notice that we actually consider the Gibbs measure of an effective(renormalized) Hamiltonian.

Other choices for the initial conditions are possible. For example,

we can take qn(0) =

√β−1k−1

n ξn. Our choice of I.C. ensures thatthe forcing term in the GLE that we will derive is mean zero (seebelow).


Hamilton’s equations of motion are:

QN + V ′(QN) =

N∑

n=1

kn(λqn − λ2QN), (189a)

qn + ω2n(qn − λQN) = 0, n = 1, . . .N, (189b)

where ω2n = kn/mn.

The equations for the heat bath particles are second order linearinhomogeneous equations with constant coefficients. Our plan isto solve them and then to substitute the result in the equations ofmotion for the Brownian particle.


We can solve the equations of motion for the heat bath variablesusing the variation of constants formula

qn(t) = qn(0) cos(ωnt) +pn(0)

mnωnsin(ωnt)

+ωnλ

∫ t

0sin(ωn(t − s))QN(s) ds.

An integration by parts yields

qn(t) = qn(0) cos(ωnt) +pn(0)

mnωnsin(ωnt) + λQN(t)

−λQN(0) cos(ωnt) − λ

∫ t

0cos(ωn(t − s))QN(s) ds.


We substitute this in equation (189) and use the initialconditions (188) to obtain the Generalized Langevin Equation

QN = −V ′(QN) − λ2∫ t

0RN(t − s)QN(s) ds + λFN(t), (190)

where the memory kernel is

RN(t) =

N∑

n=1

kn cos(ωnt) (191)

and the noise process is

FN(t) =

N∑

n=1

kn (qn(0) − λQ0) cos(ωnt) +knpn(0)

mnωnsin(ωnt)

=√β−1

N∑

n=1

√kn (ξn cos(ωnt) + ηn sin(ωnt)) . (192)


1 The noisy and random term are related through thefluctuation-dissipation theorem :

〈FN(t)FN(s)〉 = β−1N∑

n=1

kn(

cos(ωnt) cos(ωns)

+ sin(ωnt) sin(ωns))

= β−1RN(t − s). (193)

2 The noise F (t) is a mean zero Gaussian process.3 The choice of the initial conditions (188) for q, p is crucial for the

form of the GLE and, in particular, for the fluctuation-dissipationtheorem (193) to be valid.

4 The parameter λ measures the strength of the coupling betweenthe Brownian particle and the heat bath.

5 By choosing the frequencies ωn and spring constants kn(ω) of theheat bath particles appropriately we can pass to the limit asN → +∞ and obtain the GLE with different memory kernels R(t)and noise processes F (t).


Let a ∈ (0,1), 2b = 1 − a and set ωn = Naζn where ζn∞n=1 arei.i.d. with ζ1 ∼ U(0,1). Furthermore, we choose the springconstants according to

kn =f 2(ωn)

N2b ,

where the function f (ωn) decays sufficiently fast at infinity.

We can rewrite the dissipation and noise terms in the form

RN(t) =

N∑

n=1

f 2(ωn) cos(ωnt)∆ω

and

FN(t) =

N∑

n=1

f (ωn) (ξn cos(ωnt) + ηn sin(ωnt))√

∆ω,

where ∆ω = Na/N.


Using now properties of Fourier series with randomcoefficients/frequencies and of weak convergence of probabilitymeasures we can pass to the limit:

RN(t) → R(t) in L1[0,T ],

for a.a. ζn∞n=1 and

FN(t) → F (t) weakly in C([0,T ],R).

The time T > 0 if finite but arbitrary.The limiting kernel and noise satisfy the fluctuation-dissipationtheorem (193):

〈F (t)F (s)〉 = β−1R(t − s). (194)

QN(t), the solution of (190) converges weakly to the solution ofthe limiting GLE

Q = −V ′(Q) − λ2∫ t

0R(t − s)Q(s) ds + λF (t). (195)


The properties of the limiting dissipation and noise are determinedby the function f (ω).Ex: Consider the Lorentzian function

f 2(ω) =2α/πα2 + ω2 (196)

with α > 0. ThenR(t) = e−α|t|.

The noise process F (t) is a mean zero stationary Gaussianprocess with continuous paths and, from (194), exponentialcorrelation function:

〈F (t)F (s)〉 = β−1e−α|t−s|.

Hence, F (t) is the stationary Ornstein-Uhlenbeck process:

dFdt

= −αF +√

2β−1αdWdt

, (197)

with F (0) ∼ N (0, β−1).


The GLE (195) becomes

Q = −V ′(Q) − λ2∫ t

0e−α|t−s|Q(s) ds + λ2F (t), (198)

where F (t) is the OU process (197).


Q(t), the solution of the GLE (195), is not a Markov process, i.e.the future is not statistically independent of the past, whenconditioned on the present. The stochastic process Q(t) hasmemory.

We can turn (195) into a Markovian SDE by enlarging thedimension of state space, i.e. introducing auxiliary variables.

We might have to introduce infinitely many variables!

For the case of the exponential memory kernel, when the noise isgiven by an OU process, it is sufficient to introduce one auxiliaryvariable.


We can rewrite (198) as a system of SDEs:

dQdt

= P,

dPdt

= −V ′(Q) + λZ ,

dZdt

= −αZ − λP +√

2αβ−1 dWdt

,

where Z (0) ∼ N (0, β−1).

The process Q(t), P(t), Z (t) ∈ R3 is Markovian.

It is a degenerate Markov process: noise acts directly only on oneof the 3 degrees of freedom.

We can eliminate the auxiliary process Z by taking an appropriatedistinguished limit.


Set λ =√γε−1, α = ε−2. Equations (200) become

dQdt

= P,

dPdt

= −V ′(Q) +

√γ

εZ ,

dZdt

= − 1ε2 Z −

√γ

εP +

√2β−1

ε2

dWdt

.

We can use tools from singular perturbation theory for Markovprocesses to show that, in the limit as ε→ 0, we have that

1ε

Z →√

2γβ−1 dWdt

− γP.

Thus, in this limit we obtain the Markovian Langevin Equation(R(t) = γδ(t))

Q = −V ′(Q) − γQ +√

2γβ−1 dWdt

. (201)


Whenever the GLE (195) has "finite memory", we can represent itas a Markovian SDE by adding a finite number of additionalvariables.These additional variables are solutions of a linear system ofSDEs.This follows from results in approximation theory.Consider now the case where the memory kernel is a boundedanalytic function. Its Laplace transform

R(s) =

∫ +∞

0e−stR(t) dt

can be represented as a continued fraction:

R(s) =∆2

1

s + γ1 +∆2

2...

, γi > 0, (202)

Since R(t) is bounded, we have that

lims→∞

R(s) = 0.


Consider an approximation RN(t) such that the continued fractionrepresentation terminates after N steps.

RN(t) is bounded which implies that

lims→∞

RN(s) = 0.

The Laplace transform of RN(t) is a rational function:

RN(s) =

∑Nj=1 ajsN−j

sN +∑N

j=1 bjsN−j, aj , bj ∈ R. (203)

This is the Laplace transform of the autocorrelation function of anappropriate linear system of SDEs. Indeed, set

dxj

dt= −bjxj + xj+1 + aj

dWj

dt, j = 1, . . . ,N, (204)

with xN+1(t) = 0. The process x1(t) is a stationary Gaussianprocess with autocorrelation function RN(t).


For N = 1 and b1 = α, a1 =√

2β−1α we derive the GLE (198)with F (t) being the OU process (197).

Consider now the case N = 2 with bi = αi , i = 1,2 anda1 = 0, a2 =

√2β−1α2. The GLE becomes

Q = −V ′(Q) − λ2∫ t

0R(t − s)Q(s) ds + λF1(t),

F1 = −α1F1 + F2,

F2 = −α2F2 +√

2β−1α2W2,

withβ−1R(t − s) = 〈F1(t)F1(s)〉.


We can write (206) as a Markovian system for the variablesQ, P, Z1, Z2:

Q = P,

P = −V ′(Q) + λZ1(t),

Z1 = −α1Z1 + Z2,

Z2 = −α2Z2 − λP +√

2β−1α2W2.

Notice that this diffusion process is "more degenerate" than (198):noise acts on fewer degrees of freedom.

It is still, however, hypoelliptic (Hormander’s condition is satisfied):there is sufficient interaction between the degrees of freedomQ, P, Z1, Z2 so that noise (and hence regularity) is transferredfrom the degrees of freedom that are directly forced by noise tothe ones that are not.

The corresponding Markov semigroup has nice regularizingproperties. There exists a smooth density.


Stochastic processes that can be written as a Markovian processby adding a finite number of additional variables are calledquasimarkovian .

Under appropriate assumptions on the potential V (Q) the solutionof the GLE equation is an ergodic process.

It is possible to study the ergodic properties of a quasimarkovianprocesses by analyzing the spectral properties of the generator ofthe corresponding Markov process.

This leads to the analysis of the spectral properties of hypoellipticoperators.


When studying the Kac-Zwanzing model we considered a onedimensional Hamiltonian system coupled to a finite dimensionalHamiltonian system with random initial conditions (the harmonicheat bath) and then passed to the theromdynamic limit N → ∞.

We can consider a small Hamiltonian system coupled to itsenvironment which we model as an infinite dimensionalHamiltonian system with random initial conditions. We have acoupled particle-field model .

The distinguished particle (Brownian particle) is described throughthe Hamiltonian

HDP =12

p2 + V (q). (207)


We will model the environment through a classical linear fieldtheory (i.e. the wave equation) with infinite energy:

∂2t φ(t , x) = ∂2

xφ(t , x). (208)

The Hamiltonian of this system is

HHB(φ, π) =

∫ (|∂xφ|2 + |π(x)|2

). (209)

π(x) denotes the conjugate momentum field.

The initial conditions are distributed according to the Gibbsmeasure (which in this case is a Gaussian measure) at inversetemperature β, which we formally write as

”µβ = Z−1e−βH(φ,π) dφdπ”. (210)

Care has to be taken when defining probability measures ininfinite dimensions.


Under this assumption on the initial conditions, typicalconfigurations of the heat bath have infinite energy.

In this way, the environment can pump enough energy into thesystem so that non-trivial fluctuations emerge.

We will assume linear coupling between the particle and the field:

HI(q, φ) = q∫∂qφ(x)ρ(x) dx . (211)

where The function ρ(x) models the coupling between the particleand the field.

This coupling is influenced by the dipole coupling approximationfrom classical electrodynamics.


The Hamiltonian of the particle-field model is

H(q,p, φ, π) = HDP(p,q) + H(φ, π) + HI(q, φ). (212)

The corresponding Hamiltonian equations of motion are a coupledsystem of equations of the coupled particle field model.

Now we can proceed as in the case of the finite dimensional heatbath. We can integrate the equations motion for the heat bathvariables and plug the solution into the equations for the Brownianparticle to obtain the GLE. The final result is

q = −V ′(q) −∫ t

0R(t − s)q(s) + F (t), (213)

with appropriate definitions for the memory kernel and the noise,which are related through the fluctuation-dissipation theorem.


Consider now the N + 1-dimensional Hamiltonian (particle + heatbath) with random initial conditions. The N + 1− probabilitydistribution function fN+1 satisfies the Liouville equation

∂fN+1

∂t+ fN+1,H = 0, (214)

where H is the full Hamiltonian and ·, · is the Poisson bracket

A,B =

N∑

j=0

(∂A∂qj

∂B∂pj

− ∂B∂qj

∂A∂pj

).

We introduce the Liouville operator

LN+1· = −i·,H.

The Liouville equation can be written as

i∂fN+1

∂t= LN+1fN+1. (215)


We want to obtain a closed equation for the distribution function ofthe Brownian particle. We introduce a projection operator whichprojects onto the distribution function f of the Brownian particle:

PfN+1 = f , PfN+1 = h.

The Liouville equation becomes

i∂f∂t

= PL(f + h), (216a)

i∂h∂t

= (I − P)L(f + h). (216b)


We integrate the second equation and substitute into the firstequation. We obtain

i∂f∂t

= PLf − i∫ t

0PLe−i(I−P)Ls(I −P)Lf (t −s) ds+PLe−i(I−P)Lth(0).

(217)

In the Markovian limit (large mass ratio) we obtain theFokker-Planck equation (182).


L. Rey-Bellet Open Classical Systems.

D. Givon, R. Kupferman and A.M. Stuart Extracting MarcoscopicDynamics: Model Problems and Algorithms, Nonlinearity 17(2004) R55-R127.

R.M. Mazo Brownian Motion, Oxford University Press, 2002.


applied stochastic processespavl/lec1_intro.pdf · this is a basic graduate course on stochastic...

Documents