ethier s.n., kurtz t.g. markov processes characterization and convergence

Markov Processes

Markov Processes Characterization and Convergence

STEWART N. ETHIER THOMAS G. KURTZ

WILEY- INTERSCI ENCE

A JOHN WILEY & SONS, INC., PUBLICATION

Copyright 8 1986,2005 by John Wiley ti Sons, Inc. All rights reserved.

Published by John Wiley & Sons, Inc., Hoboken. New Jersey. Published simultaneously in Canada.

No part of this publication may be rcproduccd, stored in a retrieval system or transmittcd in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as pcrmittcd under Sections 107 or 108 of the 1976 United States Copyright Act, without either thc prior written permission of the Publisher, or

authorization through paymen1 of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470. Requests to the Publisher for permission should he addressed to the Permissions Deparlment, John Wiley B Sons, Inc., 111 River Strcet, Hoboken, NJ 07030, (201) 748-601 I , fax (201) 748-6008.

Limit o f Liability/Disclaimer o f Warranty: While the publisher and author have used their bcst efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties o f merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitablc for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential. or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the US. at (800) 762-2974, outside the U.S. at (3 17) 572- 3993 or fax (3 17) 572-4002.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic format. For information about Wiley products, visit our web site at www. wiley.com.

Libra y of Congress Cataloginpin-Publication is awilable.

ISBN- I3 978-0-47 1-76986-6 ISBN-I0 0-471-76986-X

Printed in the United States o f America

1 0 9 8 7 6 5 4 3 2 1

The original aim of this book was a discussion of weak approximation results for Markov processes. The scope has widened with the recognition that each technique for verifying weak convergence is closely tied to a method of characterizing the limiting process. The result is a book with perhaps more pages devoted to characterization than to convergence.

The lntroduction illustrates the three main techniques for proving convergence theorems applied to a single problem. The first technique is based on operator semigroup convergence theorems. Convergence of generators (in an appropriate sense) implies convergence of the corresponding sernigroups, which in turn implies convergence of the Markov processes. Trotter’s original work in this area was motivated in part by diffusion approximations. The second technique, which is more probabilistic in nature, is based on the martingale characterization of Markov processes as developed by Stroock and Varadhan. Here again one must verify convergence of generators, but weak compactness arguments and the martingale characterization of the limit are used to complete the proof. The third technique depends on the representation of the processes as solutions of stochastic equations, and is more in the spirit of classical analysis. If the equations “converge,” then (one hopes) the solutions converge.

Although the book is intended primarily as a reference, problems are included in the hope that it will also be useful as a text in a graduate course on stochastic processes. Such a course might include basic material on stochastic processes and martingales (Chapter 2, Sections 1-6). an introduction to weak convergence (Chapter 3, Sections 1-9, omitting some of the more technical results and proofs), a development of Markov processes and martingale problems (Chapter 4, Sections 1-4 and 8). and the martingale central limit theorem (Chapter 7, Section I ) . A selection of applications to particular processes could complete the course.

V

V i PREFACE

As an aid to the instructor of such a course, we include a flowchart for all proofs in the book. Thus, if one's goal is to cover a particular section, the chart indicates which of the earlier results can be skipped with impunity. (It also reveals that the course outline suggested above is not entirely self-contained.)

Results contained in standard probability texts such as Billingsley (1979) or Breiman (1968) are assumed and used without reference, as are results from measure theory and elementary functional analysis. Our standard reference here is Rudin (1974). Beyond this, our intent has been to make the book self-contained (an exception being Chapter 8). At points where this has not seemed feasible, we have included complete references, frequently discussing the needed material in appendixes.

Many people contributed toward the completion of this project. Cristina Costantini, Eimear Goggin, S. J. Sheu, and Richard Stockbridge read large portions of the manuscript and helped to eliminate a number of errors. Carolyn Birr, Dee Frana, Diane Reppert, and Marci Kurtz typed the manuscript. The National Science Foundation and the University of Wisconsin, through a Romnes Fellowship, provided support for much of the research in the book.

We are particularly grateful to our editor, Beatrice Shube, for her patience and constant encouragement. Finally, we must acknowledge our teachers, colleagues, and friends at Wisconsin and Michigan State, who have provided the stimulating environment in which ideas germinate and flourish. They contributed to this work in many uncredited ways. We hope they approve of the result.

STEWART N. ETHIER THOMAS G. KURTZ

Salt Lake City, Utah Madison, Wisconsin August 198s

Introduction

1 Operator Semigroups

Definitions and Basic Properties, 6 The Hille-Yosida Theorem, 10 Cores, 16 Multivalued Operators, 20 Semigroups on Function Spaces, 22 Approximation Theorems, 28 Perturbation Theorems, 37 Problems, 42 Notes, 47

2 Stochastic Processes and Martingales

1 2 3 4 5 6 7 8 9

10

Stochastic Processes, 49 Martingales, 55 Local Martingales, 64 The Projection Theorem, 71 The Doob-Meyer Decomposition, 74 Square Integrable Martingales, 78 Semigroups of Conditioned Shifts, 80 Martingales Indexed by Directed Sets, Problems, 89 Notes, 93

84

49

vii

viii CONTENTS

3 Convergence of Probability Measures

1 The Prohorov Metric, 96 2 Prohorov’s Theorem, 103 3 Weak Convergence, 107 4 Separating and Convergence Determining Sets, 11 1 5 The Space D,[O, GO), 116 6 The Compact Sets of DEIO, a), 122 7 Convergence in Distribution in &[O, m), 127 8 Criteria for Relative Compactness in DKIO, a), 132 9 Further Criteria for Relative Compactness

in D,[O, oo), 141 10 Convergence to a Process in C,[O, a), 147 11 Problems, 150 12 Notes, 154

4 Generators and Markov Processes

1 Markov Processes and Transition Functions, 156 2 Markov Jump Processes and Feller Processes, 162 3 The Martingale Problem: Generalities and Sample

Path Properties, 173 4 The Martingale Problem: Uniqueness, the Markov

Property, and Duality, 182 5 The Martingale Problem: Existence, 196 6 The Martingale Problem: Localization, 216 7 The Martingale Problem: Generalizations, 22 I 8 Convergence Theorems, 225 9 Stationary Distributions, 238

10 Perturbation Results, 253 I 1 Problems, 261 12 Notes, 273

5 Stochastic Integral Equations

1 Brownian Motion, 275 2 Stochastic Integrals, 279 3 Stochastic Integral Equations, 290 4 Problems, 302 5 Notes, 305

6 Random Time Changes

1 One-Parameter Random Time Changes, 306 2 Multiparameter Random Time Changes, 31 1 3 convergence, 321

95

155

275

306

4 Markov Processes in Zd, 329 5 Diffusion Processes, 328 6 Problems, 332 7 Notes, 335

7 Invariance Principles and Diffusion Approximations

1 The Martingale Central Limit Theorem, 338 2 Measures of Mixing, 345 3 Central Limit Theorems for Stationary Sequences, 350 4 Diffusion Approximations, 354 5 Strong Approximation Theorems, 356 6 Problems, 360 7 Notes, 364

8 Examples of Generators

1 Nondegenerate Diffusions, 366 2 Degenerate Diffusions, 371 3 Other Processes, 376 4 Problems, 382 5 Notes, 385

9 Branching Processes

1 Galton-Watson Processes, 386 2 Two-Type Markov Branching Processes, 392 3 Branching Processes in Random Environments, 396 4 Branching Markov Processes, 400 5 Problems, 407 6 Notes, 409

10 Genetic Models

I The Wright-Fisher Model, 41 1 2 Applications of the Diffusion Approximation, 41 5 3 Genotypic-Frequency Models, 426 4 Infinitely-Many-Allele Models, 435 5 Problems, 448 6 Notes, 451

11 Density Dependent Population Processes

1 Examples, 452 2 Law of Large Numbers and Central Limit Theorem, 455

337

365

386

410

452

3 Diffusion Approximations, 459 4 Hitting Distributions, 464 5 Problems, 466 6 Notes, 467

12 Random Evolutions

1 Introduction, 468 2 Driving Process in a Compact State Space, 472 3 Driving Process in a Noncompact State Space, 479 4 Non-Markovian Driving Process, 483 5 Problems, 491 6 Notes, 491

Appendixes

1 Convergence of Expectations, 492 2 Uniform Integrability, 493 3 Bounded Pointwise Convergence, 495 4 Monotone Class Theorems, 496 5 Gronwall’s Inequality, 498 6 The Whitney Extension Theorem, 499 7 Approximation by Polynomials, 500 8 Bimeasures and Transition Functions, 502 9 Tulcea’s Theorem, 504

10 Measurable Selections and Measurability of Inverses, 506 11 Analytic Sets, 506

References

Index

Flowchart

168

492

508

521

529

The development of any stochastic model involves !he identification of properties and parameters that, one hopes, uniquely characterize a stochastic process. Questions concerning continuous dependence on parameters and robustness under perturbation arise naturally out of any such characterization. In fact the model may well be derived by some sort of limiting or approximation argument. The interplay between characterization and approximation or convergence problems for Markov processes is the central theme of this book. Operator semigroups, martingale problems, and stochastic equations provide approaches to the characterization of Markov processes, and to each of these approaches correspond methods for proving convergence resulls.

The processes of interest to us here always have values in a complete, separable metric space E, and almost always have sample paths in DE(O, m), the space of right continuous E-valued functions on [O, 00) having left limits. We give DEIO, 00) the Skorohod topology (Chapter 3), under which it also becomes a complete, separable metric space. The type of convergence we are usually concerned with is convergence in distribution; that is, for a sequence of processes { X J we are interested in conditions under which limn.+m E[f(X.)J = &ff(X)] for everyfg C(D,[O, 00)). (For a metric space S, C(S) denotes the space of bounded continuous functions on S. Convergence in distribution is denoted by X, =. X . ) As an introduction to the methods presented in this book we consider a simple but (we hope) illuminating example.

For each n 2 1, define

U x ) = 1 + 3x x - - , y,(x) = 3x + + - t>(. - r>. ( 1 ) ( :>

1

Markov Processes Characterization and Convergence Edited by STEWART N. ETHIER and THOMAS G. KURTZ

Copyright 0 1986,2005 by John Wiley & Sons, Inc

2 INTRODUCTION

and let U, be a birth-and-death process in b, with transition probabilities satisfying

(2) P{ K(r + h) = j + I I ~ ( t ) a j } = n~,,(:)h + ~ ( h )

and

(3)

as Ado+. In this process, known as the sChlo8l model, x(r) represents the number of molecules at time t of a substance R in a volume n undergoing the chemical reactions

(4)

with the indicated rates. (See Chapter 11, Section 1.)

(5) x,,(t) = n’/*(n- yn(n1/2r) - 1). r 2 0.

The problem is to show that X, converges in distribution to a Markov process X to be characterized below.

The first method we consider is based on a semigroup characterization of X . Let En = {n‘/*(n-‘y - I ) : y E Z+}, and note that

1 3

3 1 Ro R, R2 + 2R S 3R,

We rescale and renormalize letting

(6) ~ w m = Erm.(t)) I x m = XJ

defines a semigroup { T,(I)} on B(E,) with generator of the form

(7) G, / ( x ) =: n3’2L,( 1 + n - ‘ / ‘x){f(x + n - ’I4) - /(x)}

+ n3/2pn( 1 + n - l / *x ) { / (x - - 3/41 - ~ ~ x ~ ~ .

(See Chapter I.) Letting A(x) = 1 + 3x2, p(x) = 3x + x3, and

(8) G~’ (x ) = 4/”(x) - x ~ ’ ( x ) ,

a Taylor expansion shows that

(9) G, f ( x ) = Gf(x) + t1”~{,4,,( I + n .- ‘/*x) - A( 1 + n -‘ l4x)}{f(x + n - ’ I* ) - / ( x ) }

+ n3/3{p,( 1 + n - ‘ l4x) - I( 1 + ~t - I/*x)} { J(X - n - 3/4) -f(x)}

+ A(1 + n - l / * x ) I’ (1 - u){f”(x + un-”*) - r ( x ) } du

for all/€ C2(R) withf‘ E Cc(R) and all x E Em. Consequently. for such/;

lim sup I G,f(x) - Gf(x) 1 = 0. n - m x c E .

Now by Theorem 1.1 of Chapter 8,

( 1 1) A E ( ( A Gf):f€ C[-00 , 001 n C’(R), G/E C[-aO, 001)

is the generator of a Feller semigroup { T(t)} on C[ - 00, 001. By Theorem 2.7 of Chapter 4 and Theorem I . I of Chapter 8, there exists a diffusion process X corresponding to (T(t)) , that is, a strong Markov process X with continuous sample paths such that

(12) ECJ(X(t)) I *.*I = - S)S(X(d)

for a l l fe C[ - 00, a03 and t 2 s 2 0. (4c: = a(X(w): u 5 s).) To prove that X , 3 X (assuming convergence of initial distributions), it

suffices by Corollary 8.7 of Chapter 4 to show that (10) holds for all / in zt core D for the generator A, that is, for all f in a subspace D of 9 ( A ) such that A is the closure of the restriction of A to D. We claim that

(13) D -= (/+ g : / I Q E C’(R),/’ E: Cc(W), (x’g)‘ E Cc(W)}

is a core, and that (10) holds for all/€ D. To see that D is a core, first check that

(14) ~ ( A ) = ( J E C [ - C Q , ~ ] n C 2 ( R ) : f ” ~ ~ ( W ) , x 3 f ’ ~ C [ - o o , o o ] } .

Then let h E C;(R) satisfy xI - 5 h s f E 9 ( A ) , choose g E: D with (x’g)’ E Cc(W) and x 3 ( f - g)’ E e(R) and define

(15)

Thenj,, + g E D for each m,f , + g -+f, and G(fm + Q)-+ C/.

a martingale problem. Observe that

and put h,(x) = h(x/m). Given

SdX) = S(0) - do) + (j - gY( Y )hm( Y 1 d ~ . s: The second method is based on the characterization of X as the solution of

is an {.Ffn)-martingale for each /E B(E,) with compact support. Conse- quently, if some subsequence {A’,,,) converges in distribution to X , then, by the continuous mapping theorem (Corollary 1.9 of Chapter 3) and Problem 7 of Chapter 7,

4 ~ O W c r I o N

is an {Pf)-martingale for eachfe C,'(R), or in other words, X is a solution of the martingale problem for { ( A G f ) : f c C,'(W)}. But by Theorem 2.3 of Chapter 8, this property characterizes the distribution on Dn[O, 00) of X . Therefore, Corollary 8.16 of Chapter 4 gives X, = X (assuming convergence of initial distributions), provided we can show that

Let (p(x) I ex + e-x, and check that there exist constants C , , a O such.that C,,a < G,cp I; C,,,rp on [-a, u] for each n 2 I and ct > 0, and Ka+-

00. Letting = inf ( f 2 0: I X,,(t) I 2 a}, we have

I inf C P ( Y ) ~ SUP Ixn(t)lk a { ostsr e -G. 4 T

Irl L a (1 9)

ELeXP - Cn, a(?n, 8 A 73) cp(Xn(Tn, a A VJ 5 QdXn(O))l

by Lemma 3.2 of Chapter 4 and the optional sampling theorem. An additional (mild) assumption on the initial distributions therefore guarantees (1 8).

Actually we can avoid having to verify (18) by observing that the uniform convergence of G, f to Gf for f e C:(R) and the uniqueness for the limiting martingale problem imply (again by Corollary 8.16 of Chapter 4) that X , =. X in Dad[O, 00) where WA denotes the one-point compactification of R. Con- vergence in &LO, 00) then follows from the fact that X, and X have sample paths in DRIO, 00).

Both of the approaches considered so far have involved characterizations in terms of generators. We now consider methods based on stochastic equations. First, by Theorems 3.7 and 3.10 of Chapter 5, we can characterize X as the unique solution of the stochastic integral equation

where W is a standard one-dimensional, Brownian motion. (In the present example, the term 2JW(t) corresponds to the stochastic integral term.) A convergence theory can be developed using this characterization of X, but we do not do so here. The interested reader is referred to Kushner (1974).

The final approach we discuss is based on a characterization of X involving random time changes. We observe first that U, satisfies

where N, and N - are independent, standard (parameter I), Poisson processes. Consequent I y , X, satisfies

X,(r) = X,(O) + n- 3 /4R+ ( n 3/2 A,( I + n - ' /*X,(s)) ds (22)

- n-"'R.(nl" 6'p,(l + n-'/4X,(s)) d s )

+ n3l4 [ ( A , - p&I + n - ''4X,(s)) ds,

where R + ( u ) = N + ( u ) - u and R _ ( u ) = N-(u) - u are independent, centered, standard, Poisson processes. Now i t i s easy to see that

(23) ( n ' /*R + (n3/2 * 1, n 'l4R - (n3'2 .)) =. ( W+ , W- 1, where W+ and W- are independent, standard, one-dimensional Brownian motions. Consequently, i f some subsequence {A'".) converges in distribution to X, one might expect that

X ( t ) = X(0) + W+(4t) + W ( 4 t ) - X ( S ) ~ ds. (24) s.' (In this simple example, (20) and (24) are equivalent, but they wi l l not be so in general.) Clearly, (24) characterizes X, and using the estimate (18) we conclude X, - X (assuming convergence of initial distributions) from Theorem 5.4 of Chapter 6.

For a further discussion of the Schlogl model and related models see Schlogl (1972) and Malek-Mansour et al. (1981). The martingale proof of convergence is from Costantini and Nappo (1982), and the time change proof i s from Kurtz(1981c).

Chapters 4-7 contain the main characterization and convergence results (with the emphasis in Chapters 5 and 7 on diffusion processes). Chapters 1-3 contain preliminary material on operator semigroups, martingales, and weak convergence, and Chapters 8- I 2 are concerned with applications.

1

Operator semigroups provide a primary tool in the study of Msrkov processes. In this chapter we develop the basic background for their study and the existence and approximation results that are used later as the basis for existence and approximation theorems for Markov processes. Section 1 gives the basic definitions, and Section 2 the Hille-Yosida theorem, which characterizes the operators that are generators of semigroups. Section 3 concerns the problem of verifying the hypotheses of this theorem, and Sections 4 and 5 are devoted to generalizations of the concept of the generator. Sections 6 and 7 present the approximation and perturbation resuJts.

Throughout the chapter, L denotes a real Banach space with norm 11 * 11.

OPERATOR SEMICROUPS

1. DEFINITIONS AND BASIC PROPERRES

A one-parameter family { T(t): t 2 0) of bounded linear operators on a Banach space L is called a semigroup if T(0) = I and T(s + t ) = T(s)T(c) for all s, t 2 0. A semigroup (T(t)) on L is said to be strongly continuous if lim,,o T(r)/ =/for everyfe L; it is said to be a contraction semigroup if 11 T(t)II 5 1 for all t 2 0.

Given a bounded linear operator B on L, define



1. DmNmoNz AND EASIC ?ROPERTIES 7

A simple calculation gives e'"')' = e""e'' for all s, t 2 0, and hence {e'"} is a semigroup, which can easily be seen to be strongly continuous. Furthermore we have

An inequality of this type holds in general for strongly continuous sernigroups.

1.1 Proposition there exist constants M 2 1 and o 2 0 such that

(1 -3) II T(t)lI 5 Me"', t 2 0.

Let (T(t)) be a strongly continuous semigroup on L. Then

Proof. Note first that there exist constants M 2 I and ro > 0 such that 11 T(t) 11 5 M for 0 I t s t o . For if not, we could find a sequence (t,} of positive numbers tending to zero such that 11 T(t,)(( -+ 00, but then the uniform boundedness principle would imply that sup,(( T(rJfI1 = 00 for some f E L, contradicting the assumption of strong continuity. Now let o = t i log M. Given t 2 0, write t = kt, + s, where k is a nonnegative integer and 0 s s < t,; then

( 1.4) 0 I( T(t)I( = II 'f(~)T(t,,)~Il s MM' r; MM'/'O = Me"'.

1.2 Corollary each$€ L, t -+ T(t)/is a continuous function from [0, 00) into L.

Let { T(r)) be a strongly continuous semigroup on L. Then, for

1.3 Remark Let { T(r)} be a strongly continuous semigroup on L such that (1.3) holds, and put S(t) = e-"'T(r) for each t 2 0. Then {S(t)) is a strongly continuous semigroup on L such that

(1.7) II W II s M, t 2 0.

8 OraATORS€MIGROWS

In particular, if M = 1, then {S(t)} is a strongly continuous contraction semigroup on L.

Let {S(t)} be a strongly continuous semigroup on L such that (1.7) holds, and define the norm 111 111 on L by

Then 11f11 5; I I I J I I I 5; Mllfll for each f E L, so the new norm is equivalent to the original norm; also, with respect to 111 * 111, {S(t)) is a strongly continuous contraction semigroup on L.

Most of the results in the subsequent sections of this chapter are stated in terms of strongly continuous contraction semigroups. Using these reductions, however, many of them can be reformulated in terms of noncontraction semigroups. 0

A (possibly unbounded) linear operator A on L is a linear mapping whose domain 9 ( A ) is a subspace of L and whose range a ( A ) lies in L. The graph of A is given by

Note that L x L is itself a Banach space with componentwise addition and scalar multiplication and norm [ l ( J @)[I = llfll + IIg 11. A is said to be closed if 9 ( A ) is a closed subspace of L x L.

The (injinitesimal) generator of a semigroup { T(c)) on L is the linear operator A defined by

(1.10) 1

A , = lim ; { T ( t ) f - J } . 1-0

The domain 9 ( A ) of A is the subspace of allJE L for which this limit exists. Before indicating some of the properties of generators, we briefly discuss the

calculus of Banach space-valued functions. Let A be a closed interval in ( - 00, a), and denote by CJA) the space of

continuous functions u : A+ L. Let Cl(A) be the space of continuously differentiable functions u : A + L.

If A is the finite interval [a, b] , u : A + L is said to be (Rietnann) integrable over A if limd,, u(sk) ( fk - t,,- I ) exists, where a = to S s, 5 I l I . . 5;

t,- , s s, s f n = b and S = max ( r r - f k - l); the limit is denoted by jb, u(t)dt or u(t)dt. If A = [a, a), u : A + L is said to be integrable over A if u I , ~ , ~ , is

integrable over [a, b] for each b 2 a and limg,, Jt u(t) dt exists; again, the limit is denoted by {A ~ ( t ) dt or {; u(r) dt.

We leave the proof of the following lemma to the reader (Problem 3).

1. MflMTlONS AND 8ASlC PROPERTIES 9

1.4 Lemma (a) If u E C,jA) and J A l l u(t ) I1 dt < 00, then u is integrable over A and

(1 .1 I )

In particular, if A is the finite interval [a, 61, then every function in C,(A) is integrable over A.

Let B be a closed linear operator on L. Suppose that u E CJA), u(t) E 9 ( E ) for all t E A, Bu E CJA), and both u and Bu are integrable over A. Then JA U ( t ) dt E 9 ( B ) and

(1.12)

(c) If u E Ci,[a, b] , then

(b)

B u(t) dt Bu(t) dr. I =I (1.13) I' $ u(t) dt = u(b) - u(a).

1.5 Proposition Let (T( t ) } be a strongly continuous semigroup on L with generator A.

(a) I f f € L and t 2 0, then So T ( s ) f d s E 9 ( A ) and

(1.14)

(b)

(1.15)

(c) I f f€ 9 ( A ) and r 2 0, then

I f f € 9 ( A ) and t 2 0. then T ( t ) / E B(A) and

d -- r(t)j= A T ( t ) / = T(r)AJ dt

(1.16) T(t)J - j = A T(.s)j ds = T(s)Af ds.

Proof. (a) Observe that

for all h > 0, and as h -, 0 the right side of(I.17)converges to T ( t ) / - f :

10 OPERATOR SEMlGROUPS

(b) Since

(1.18)

for all h > 0, where A, = h-'[T(h) - I ] , it follows that T ( t ) f e 9 ( A ) and (d /d t )+ T(t)f = A T(r)/ = T(t)A$ Thus, it sufices to check that (d /d f ) - T(r) f -- T(r)Af (assuming t > 0). But this follows from the identity

(1.19) 1 - - h - h ) f - W)SI - T(t)A/

= T(t - h)[A, - A]f+ [T(I - h) - T(t)]Af,

valid for 0 < h 5 t .

(c) This is a consequence of (b) and Lemma 1.4(c). 0

1.6 Corollary If A is the generator of a strongly continuous semigroup { T(t)} on L, then 9 ( A ) is dense in L and A is closed.

Proof. Since Iim,,o + t - ' fo T(s)f ds = f for every f c L, Proposition 1 .qa) implies that 9 ( A ) is dense in L. To show that A is closed, let {f ,} c 9 ( A ) satisfy $, 4 f and AS,- g. Then T(r)f, -Jn = ro T(s)AJn ds for each t > 0, so, letting n-+ a, we find that T(r) f - f = 6 T(s)g ds. Dividing by t and letting

0 I-+ 0, we conclude that je 9 ( A ) and Af= g.

2. THE HILL€-YOSIDA THEORfM

Let A be a closed linear operator on L. If, for some real 2, A - A ( K A1 - A ) is one-to-one, W ( l - A ) = L, and (1 - A)- ' is a bounded linear operator on L, then 1 is said to belong to the resoluent set p(A) of A, and RA = (A - A)- ' is called the resoluenr (at A) of A.

2.1 Proposition Let { T ( I ) ) be a strongly continuous contraction semigroup on L with generator A. Then (0, 00) c p(A) and

(2.1) (A - A) - 'g = e-A'T(tb dr

for all g E L and d > 0.

Proof. Let 1 > 0 be arbitrary. Define U, on L by U A g = J$ e-"T(t)g df. Since

(2.2)

0)

It U ~ g l l Lrn e-"'l/ T(r)sll df 9 ~- ' l l g l l

2. THE HILLL-YOSIDA THEOREM 11

for each g E L, U A is a bounded linear operator on L. Now given g E L,

for every h > 0, so, letting h-, 0,. we find that UAg E g ( A ) and AUAg = AU,g - g, that is,

(2.4) (1 - A)UAg 9, 9 E L.

In addition, if g E $@(A), then (using Lemma 1.4(b))

(2.5) UAAg = e- "T( t )Ag dt = [ A(e-"T(t )g) dt

= A lm e-"'(t)g dt = AuAg,

so

(2.6) uA(A - A)g = 99 g E %A).

By (2.6), A - A is one-to-one, and by (2.4), 9 ( A - A) = L. Also, (A - A) - ' = U A by (2.4) and (2.6), so A E p(A). Since rl > 0 was arbitrary, the proof is complete. 0

Let A be a closed linear operator on L. Since (A - A)(p - A ) = (p - AHA - A ) for all A, p E p(A), we have (p - A)- ' (A - A) . . ' = ( A - A)- - ' (p - A ) I , and a simple calculation gives the resolvent identity

(2.7) RA R , = R, RA = (A - p ) - ' ( R , - RA), A, p E p(A).

IfI.Ep(A)andJA-pI < I)R,II-',then

(2.8)

defines a bounded linear operator that is in fact (p - A ) - ' . In particular, this implies that p(A) is open in R.

A linear operator A on L is said to be dissipative if II J j - AjII 2 A l l f l l for every/€ B ( A ) and I > 0.

2.2 lemma Let A be a dissipative linear operator on L and let 1 > 0. Then A is closed if and only if #(A - A ) is closed.

Proof. Suppose A is closed. If (1;) c 9 ( A ) and (A - A)jw-+ h, then the dissipativity of A implies that {J.} is Cauchy. Thus, there exists/€ L such that

12 OPERATORSEMICRourS

L.+J and hence Al,,--+ Af - h. Since A is closed,fe 9 ( A ) and h = (A - A)J It follows that @(I - A ) is closed.

Suppose *(A - A) is closed. If {L} c 9 ( A ) , S,-J and A h 3 g, then (A - A)fn -+ ?/- g, which equals (A - A)J, for somefo E 9 ( A ) . By the dissipativity of A,

0 f n d f o , and hence/=fO E 9 ( A ) and As= g. Thus, A is closed.

2.3 lemma Let A be a dissipative closed linear operator on L, and put p+(A) = p(A) n (0, 00). If p + ( A ) is nonempty, then p+(A) = (0, a).

froof. I t suffices to show that p+(A) is both open and closed in (0, a). Since &A) is necessarily open in R, p + ( A ) is open in (0, 00). Suppose that {i"} c p+(A) and A,-+ A > 0. Given g E L, let g,, = (A - AKA, - A ) - ' g for each ti, and note that, because A is dissipative,

(2.9) lim IIg,, - g 11 = lim 11 (I - Am)& - A ) - ' g 11 5 lim 1.1-1.1 11 g 11 = 0.

Hence @(A - A ) is dense in L, but because A is closed and dissipative, 9 ( A - A) is closed by Lemma 2.2, and therefore @(A - A) = L. Using the dissipativity of A once again, we conclude that I - A is one-to-one and II(A - A)-'(I s I - ' . I t follows that 1 B p+(A), so p + ( A ) is closed in (0, a), as

I -al *-.OD n-al 4

required. 0

2.4 lemma Let A be a dissipative closed linear operator on L, and suppose that 9 ( A ) is dense in L and (0, 03) c p(A). Then the Yosida approximation A, of A, defined for each A > 0 by A, = RA(A - A ) - ' , has the following properties:

la) For each A > 0, Al is a bounded linear operator on L and {PJ} is a

(b) A, A, = A, A, for all A, p > 0.

(c) lim,-m A, f = Affor everyfe 9 ( A ) .

strongly continuous contraction semigroup on L.

Proof. ( I - A)R, = I on L and R,(A - A ) = I on $+I), it follows that

(2.10) A , = A ' R , - A l on L, A > O ,

and

For each R > 0. let R, = (A - A)- ' and note that 11 R , 11 5 A - I . Since

2. T M HILL€-YOSIDA THEOREM 13

for all t 2: 0, proving (a). Conclusion (b) is a consequence of (2.10) and (2.7). As for (c), we claim first that

(2.1 3) lim I R , f = f , SE L. d-+m

Noting that l l L R a f - l l l = II RAAfll s A-'I(A/II 4 0 as A+ a, for each f e 9 ( A ) , (2.13) follows from the facts that 9 ( A ) is dense in L and lll.Ra - I l l S 2 for all 1 > 0. Finally, (c) is a consequence of (2.1 I ) and (2. I 3). 0

2.5 lemma If B and C are bounded linear operators on L such that BC = CB and 11 elB (I I; I and 11 efc 11 5 I for all t 1 0, then

(2.14) II e"!f - elC/ It I t I t Bf - C/ I1 for everyfe L and t 2 0.

Proof. The result follows from the identity

= [ e'"e''- B - C)f ds.

(Note that the last equality uses the commutivity of B and C.) 0

We are now ready to prove the Hille-Yosida theorem.

2.6 Theorem A linear operator A on L is the generator of a strongly continuous contraction semigroup on L if and only if:

(a) 9 ( A ) is dense in L. (b) A is dissipative. (c) a(1 - A ) = L for some R > 0.

Proof. The necessity of the conditions (a)+) follows from Corollary 1.6 and Proposition 2.1. We therefore turn to the proof of sulliciency.

By (b), (c), and Lemma 2.2, A is closed and p(A) n (0, m) i s nonempty, so by Lemma 2.3, (0, m) c p(A). Using the notation of Lemma 2.4, we define for each L > 0 the strongly continuous contraction semigroup {T'(c)} on L by K(t ) = erAA. By Lemmas 2.4b) and 2.5,

(2.16) II nw- q(t)/ll 111 AJ- AJll

14 OrUATOROMCROUIS

for all f~ L, t 2 0, and A, p > 0. Thus, by Lemma 2.4(c), limA*m T,(t)/exists for - all t 2 0, uniformly on bounded intervals, for allfe 9 ( A ) , hence for every f~ B(A) = L. Denoting the limit by T(t)fand using the identity

(2.17) T(s + t ) j - T(s)T(t)f= [T(s + r ) - T, (s + t)Jf

+ T,(s)CT,(t) - 7'(01S+ CT,(s) - WJWJ; we conclude that { T(t)} is a strongly continuous contraction semigroup on L.

I .5(c), It remains only to show that A is the generator of {T(t)} . By Proposition

(2.18)

for altfE L, t 2 0, and R > 0. For eachfE 9 ( A ) and r 2 0, the identity

(2.19) together with Lemma 2 4 4 , implies that G(s)AJ-r T(s)Af as A+ bc), uniformly in 0 5 s s t. Consequently, (2.18) yields

T,(s)A s - T(s)Af = T*(sXAJ - Af) + c TAW - 7wl A/;

(2.20)

for all/€ 9 ( A ) and t 2 0. From this we find that the generator B of { T(r)} is an extension of A. But, for each 1 > 0,A - B is one-to-one by the necessity of (b), and #(A - A ) = L since rl E p(A). We conclude that B = A, completing the proof. 0

The above proof and Proposition 2.9 below yield the following result as a by-product.

2.7 Proposition Let { T(t)} be a strongly continuous contraction semigroup on L with generator A, and let Ad be the Yosida approximation of A (defined in Lemma 2.4). Then (2.21)

so, for each f E L, liniA-,m e'"1/= T(r)f for all I 2 0, uniformly on bounded intervals.

1Ie'"Y- T(t)fII 5 tit As- AfII, fs %4), t & 0, rt > 0,

2 8 Corollary Let {T(r)} be a strongly continuous contraction semigroup on L with generator A. For M c L, let (2.22) Ay i= { A > 0: A(A - A)- ' : M 4 M}. If either (a) M is a closed convex subset of L and AM is unbounded, or (b) M is a closed subspace of L and AM is nonempty, then (2.23) T(t): M-+ M, t 2 0.

1. TH€ HNLE-VOSIDA THEOREM 1s

Proof. If A, j~ > 0 and I 1 - p/ l I < I, then (cf. (2.8))

(2.24) p ( p - A ) - ' = n = O f ;(* - $ [ A ( I - A ) - 1 ] " ' ?

Consequently, if M is a closed convex subset of L, then I E AM implies (0, A] c AM, and if M is a closed subspace of L, then A. E AM implies (0, 2 4 t A,,, . Therefore, under either (a) or (b), we have AM = (0, 00). Finally, by (2.10).

(2.25) exp { IA , } = exp { - t I ) exp { t A [ l ( l t - A ) - ' ] )

for all I 2 0 and I > 0, so the conclusion follows from Proposition 2.7. 0

2.9 Proposition Let { T(t)} and {S(t)} be strongly continuous contraction semigroups on L with generators A and B, respectively. If A = B, then T(t) = S(t) for all r 2 0.

Proof. This result is a consequence of the next proposition. 0

2.10 Proposition Let A be a dissipative linear operator on L. Suppose that u : [0, a)-+ L is continuous, ~ ( t ) E Q(A) for all r > 0, Au: (0, a)-+ L is continuous, and

(2.26) u(t) = U(E) + Au(s) ds,

for all t > E > 0. Then II u(r) II 5 II 40) It for all t 2 0.

16 OPERATOR SEMlCROUrS

where the first inequality is due to the dissipativity of A. The result follows from the continuity of Au and u by first letting max (t, - t i - ,)+ 0 and then letting c+ 0. 0

In many applications, an alternative form of the Hille-Yosida theorem is more useful. To state it, we need two definitions and a lemma.

A linear operator A on L is said to be closable if it has a closed linear extension. If A is closable, then the closure A of A is the minimal closed linear extension of A ; more specifically, it is the closed linear operator 6 whose graph is the closure (in L x L) of the graph of A.

2.11 lemma Let A be a dissipative linear operator on L with 9 ( A L. Then A is ciosable and L@(A - A ) = 9?(A - A )for every I > 0.

dense in

Proof. For the first assertion, it suffices to show that if {A} c 9 ( A ) , 0, and Af, -+g E L, &hen g = 0. Choose {g,} c $(A) such that g,,,--tg. By the dissipativity of A,

(2.28) IIV - - 4 It = lim I I (A- A h , + &)I1 a - m

2 lim A I l g m + KII AI IgmI I n- m

for every 1 > 0 and each m. Dividing by I and letting A+ 00, we find that IIg, - g II 2 II g, II for each m. Letting m--, 00, we conclude that g = 0.

Let 1 > 0. The inclusion @(A - A) =)@(A - A) is obvious, so ro prove equality, we need only show that 5?(I - A) is closed. But this is an immediate consequence of Lemma 2.2. 0

2.12 Theorem A linear operator A on L is closable and its closure A is the generator of a strongly continuous contraction semigroup on L if and only i f

(a) 9 ( A ) is dense in L. (b) A is dissipative. (c) B(1- A) is dense in L for some A > 0.

Proof. By Lemma 2.1 1, A satisfies (a)-+) above if and only if A is closable and A’ satisfies (a)+) of Theorem 2.6. a

3. CORES

In this section we introduce a concept that is of considerable importance in Sections 6 and 7.

Let A be a closed linear operator on L. A subspace D of 9 ( A ) is said to be a core for A if the closure of the restriction of A to D is equal to A (i.e., if A J , = A). -

3.1 Proposition Let A be the generator of a strongly continuous contraction semigroup on L. Then a subspace D of 9 ( A ) is a core for A if and only if D is dense in L and w(1. - AID) is dense in L for some 1 > 0.

3.2 Remark A subspace of L is dense in L if and only if it is weakly dense (Rudin (l973), Theorem 3.12). 0

Proof. The sufficiency follows from Theorem 2.12 and from the observation that, if A and B generate strongly continuous contraction semigroups on L and if A is an extension of 8, then A = B. The necessity depends on Lemma 2.1 1. 0

3.3 Proposition Let A be the generator of a strongly continuous contraction semigroup IT([)} on L. Let Do and D be dense subspaces of L with Do c D c 9 ( A ) . (Usually, Do = D.) If T(r): Do-+ D for all t 2 0, then D is a core for A.

Proof. Given f E Do and L > 0,

(3.1)

for n = I , 2,. . .. By the strong continuity of { T(t ) } and Proposition 2.1,

(3.2) I lim (i. - A)S, = lim - e' ak/n7(:)(,l - A)/

n - m n-(u k = O

= lm e -"T(t)(d - A)$&

= (1 - A ) - ' ( L - A)!=/ :

so a(>. - A I D ) 3 Do. This sufices by Proposition 3. I since Do is dense in L. 0

Given a dissipative linear operator A with 9 ( A ) dense in L, one often wants to show that A generates a strongly continuous contraction semigroup on L. By Theorem 2.12, a necessary and sufficient condition is that .%(A - A ) be dense in L for some A > 0. We can view this problem as one of characterizing a core (namely, g ( A ) ) for the generator of a strongly continuous contraction semigroup, except that, unlike the situation in Propositions 3.1 and 3.3, the generator is not provided in advance. Thus, the remainder of this section is primarily concerned with verifying the range condition (condition (c)) of Theorem 2.12.

Observe that the following result generalizes Proposition 3.3.

18 OrUATOR YMIGROUK

3.4 Propositlon Let A be a dissipative linear operator on L, and Do a subspace of B(A) that is dense in L. Suppose that, for eachJE Do, there exists a continuous function u,: [O, 00)" L such that u,(O) =1; u,(t) E .@(A) for all r > 0, Au,: (0, a)-+ L is continuous, and

(3.3)

for all t > E > 0. Then A is closable, the closure of A generates a strongly continuous contraction semigroup { T ( f ) } on L, and T(t)J = u,(t) for all f E Do and r 2 0.

Proof. By Lemma 2.11, A is closable. Fix f~ Do and denote uf by u. Let to > E > 0, and note that I:" e-'u(t) dt E 9(A) and

(3.4) 2 lo e-'u(t) dt = e-'Au(t) At.

Consequently,

(3 .5)

I0

I'" e-'u(r) dt = (e-a - e-'O)u(c) + lo e-' [ Au(s) ds dt

= (e-'- e-'O)u(c) +

= A I'" e 3 ( t ) dt + e-'u(c) - e-'Ou(t,).

(e-# - e-'O)Au(s) ds I'" Since IIu(t)(l 5 l l f l l for all t 2 0 by Proposition 2.10, we can let 6-0 and to -+ Q) in (3.5) to obtain $; e-'u(t) dr E B(2) and

(3.6) ( I - 2) im e-'u(t) dr =J:

We conclude that @(l - 2) 3 Do, which by Theorem 2.6 proves that 2 generates a strongly continuous contraction semigroup { T(r)} on L. Now for each f E D o .

(3.7) W f - W f = I' m4m

for all t > E > 0. Subtracting (3.3) from this and applying Proposition 2.10 0 once again, we obtain the second conclusion of the proposition.

The next result shows that a suficient condition for A' to generate is that A be triangulizable. Of course, this is a very restrictive assumption, but it is occasionally satisfied.

3. CORES 19

3.5 Proposition Let A be a dissipative linear operator on L, and suppose that L,, L,, L 3 , . . . is a sequence of finite-dimensional subspaces of 9 ( A ) such that u."- , L, is dense in L. If A : L , 4 L, for n = I , 2, . . ., then A is closable and the closure of A generates a strongly continuous contraction semigroup on L.

Proof. For n = 1, 2, . . ., ( A - AWL,) L, for all 1 not belonging to the set of eigenvalues of AIL., hence for all but at most finitely many L > 0. Conse- quently, ( A - AWU,", , L,) = u:=, L, for all but at most countably many L > 0 and in particular for some A > 0. Thus, the conditions of Theorem 2.12 are satisfied. C3

We turn next to a generalization of Proposition 3.3 in a different direction. The idea is to try to approximate A sufficiently well by a sequence of generators for which the conditions of Proposition 3.3 are satisfied. Before stating the result we record the following simple but frequently useful lemma.

3.6 Lemma Let A , , A 2 , . . I and A be linear operators on L, Do a subspace of L, and A > 0. Suppose that, for each g E D o , there existsJ, E g(A,)nd(A) for n = 1.2,. . .such that g, = ( A - A,)f,+gasn-+ 60 and

lim [ [ (A , - A)Ll[ = 0. n-.m

(3.8)

Then *(A - A) 3 D o .

Proof. Given g E Do, choose {f,} and {g,} as in the statement of the lemma, and observe that limn-m II(A - A)J, - g,II -- 0 by (3.8). It follows that

0 limn+m I( ( A - A)f, - g 11 = 0, giving the desired result.

3.7 Proposition Let A be a linear operator on L and Do and D, dense subspaces of L satisfying Do c 9 ( A ) c D, c L. Let 111 . 111 be a norm on D , . For n = 1,2, . . . , suppose that A, generates a strongly continuous contraction semigroup IT&)) on L and d ( A ) c O(A,). Suppose further that there exist w 2 0 and a sequence {&,} c (0, 60) tending to zero such that, for n = 1.2, . . . ,

and

(3.1 1) T,(t): Do+ 9 ( A ) , r 2 0.

Then A is closable and the closure of A generates a strongly continuous contraction semigroup on L.

20 OPERATOISMCROUPS

Proof. Observe first that O(A) is dense in L and, by (3.9) and the dissipativity of each A,, A is dissipative. It therefore sufices to verify condition (c) of Theorem 2.12.

Fix 1 > o. Given g E D o , let

(3.12)

for each m, n 2 1 (cf. (3.1)). Then, for n = 1, 2, . . ., ( A - An)fm,,-+ e-''T(f)(A - An)g dt = g as m - r 00, so there exists a sequence {m,f of

positive integers such that ( A - A,,)S,,,-+ gas n--, 03. Moreover,

(3.13) It (An -. Alfm., n II 111 fm. n 111 M 2

k = O Ill g 111 5 en m,- 1 C e- Wa&h

- 0 as n+m

by (3.9) and (3.10), so Lemma 3.6 gives the desired conclusion. 0

3.8 Corollary Let A be a linear operator on L with B(A) dense in L, and let Ill * 111 be a norm on 9 ( A ) with respect to which 9 ( A ) is a Banach space. For n = 1, 2, . . ., let T. be a linear 11 ))-contraction on L such that T,: 9 ( A ) - + 9 ( A ) , and define A, = n(T, - I). Suppose there exist w 2 0 and a sequence {t,} c (0, a) tending to zero such that, for n = 1, 2, . . ., (3.9) holds and

(3.14)

Then A is closable and the closure of A generates a strongly continuous contraction semigroup on L.

Proof. We apply Proposition 3.7 with Do = D , = 9 ( A ) . For n = I , 2,. , ., exp (t.4,) : 9 ( A ) + 9 ( A ) and

(3.15) 111 ~ X P (tAn) I m A ) 111 S ~ X P { -nil exp {nt 111 T. ( @ ( A ) 111 f s ~ X P {all for all t 2 0, so the hypotheses of the proposition are satisfied. 0

4. MULTlVAlUED OPERATORS

Recall that if A is a linear operator on L, then the graph g ( A ) of A is a subspace of L x L such that (0, g) E g(A) implies g = 0. More generally, we regard an arbitrary subset A of L x L as a multiualued operator on L with domain 9 ( A ) = {/: (J g) E A for some g } and range * ( A ) = (g: (JI g ) e A for some/}. A c L x L is said to be linear if A is a subspace of L x L. I f A is linear, then A is said to be sinyfe-uaiued if (0, g ) E A implies g = 0; in chis case,

4. MULTIVALUED OPERATORS 21

A is a graph of a linear operator on L, also denoted by A, so we write Af = g if (J g) E A. If A c L x L is linear, then A is said to be dissipariue if (I lf - g II 2 R (I.fII for all (5 g ) E A and R > 0 ; the closure A’ of A is of course just the closure in L x L of the subspace A. Finally, we define 1 - A = ((J A f - g ) : (J g ) E A } for each 1 > 0.

Observe that a (single-valued) linear operator A is closable if and only if the closure of A (in the above sense) is single-valued. Consequently. the term “closable” is no longer needed.

We begin by noting that the generator of a strongly continuous contraction semigroup is a maximal dissipative (multivalued) linear operator.

4.1 Proposition Let A be the generator of a strongly continuous contraction semigroup on L. Let B c L x L be linear and dissipative, and suppose that A c 8. Then A = B.

Proof. Let U; g ) E B and 1 > 0. Then ( f . 1. - g ) E I - B. Since A E p(A), there exists h E 9 ( A ) such that Ah - Ah = AJ- g. Hence (h, If-- g ) E

1 - A c A - B. By linearity, (1- h, 0 ) E I - B, so by dissipativity, J = h. 0 Hence g = Ah, so ( J ; g) E A.

We turn next to an extension of Lemma 2.1 1.

4.2 Lemma Let A t L x L be linear and dissipative. Then - (4.1) A0 = {(SI 8 ) E A’: 9 E @ A ) }

is single-valued and cR(A - A ) = 9(1 - A) for every 1 > 0.

Proof. Given (0, g) E A,, we must show that g = 0. By the definition of A,, there exists a sequence {(g., h,)] c A such that g,-+g. For each n, (g,, h, + l ,g) E A by the linearity of A, so II Ag, - h,, - Ag I1 2 dII g, II for every 1. > 0 by the dissipativity of A’. Dividing by 1 and letting A - a, we find that Ilg,, - gll 2 )lg. I1 for each n. Letting n-, a, we conclude that g = 0.

The proof of the second assertion is similar to that of the second assertion of Lemma 2. I I . 0

The main result of this section is the following version of the Hille-Yosida theorem.

4.3 Theorem Let A c L x L be linear and dissipative, and define A. by (4.1). Then A. is the generator of a strongly continuous contraction semigroup on 9 ( A ) if and only if 9?(R - A ) 2 9 ( A ) for some A > 0. -

Proof. A, is single-valued by Lemma 4.2 and is clearly dissipative, so by the Hille-Yosida theorem (Theorem - 2.6), A, generates a strongly continuous - contraction semigroup on 9 ( A ) if and only if 9 ( A , ) is dense in 9 ( A ) and @(I. - A,) = 9 ( A ) for some A > 0. The latter condition is clearly equivalent to

22 OPERATOR SEMIGROUPS

9 ( L - A) =3 a(A) for some A > 0. which by Lemma 4.2 is equivalent to 41(1 - A ) 3 d(A) for some 1 - > 0. Thus, to complete the proof, - it suffices to show that 9 ( A o ) is dense in 9 ( A ) assuming that 5?(A - A,) = B(A) for some 1 > 0.

By Lemma 2.3, Se(1 - A,)= 9 ( A ) for every A >O, so 9(1 - A ) = 9 ( R - A) 3 9 ( A ) for every R > 0. By the dissipativity of A, we may regard (A - A)-' as a (single-valued) bounded linear operator on .@(A - A) of norm at most L- ' for each 1 > 0. Given cf; g) E A' and R > 0, Af - g e @R - A) and /E 9 ( X ) c 9 ( A ) c W(A - A), so g E g(A- X), and therefore II A(d - A)-'f--/Il = II(A - A)-'gll 5 1-'IIgII. Since 9 ( A ) is dense in O(A),it follows that

(4.2)

- -

-

- lim A(L - A)-y=S, f E 9 ( ~ ) . I - m

(Note that this does not follow from - (2.13).) But clearly, (A - A)- ' : &(A - A0)+ 9 ( A o ) , that is, ( A - A)- ': 9 ( A ) - + 9(Ao) , for all L > 0. In view

0 of (4.2), this completes the proof.

Milltivalued operators arise naturally in several ways. For example, the following concept is crucial in Sections 6 and 7.

For n = 1, 2, . . ., let L,, in addition to L, be a Banach space with norm also denoted by 11 * 11, and let n,: L-. L, be a bo'unded linear transformation. Assume that sup, IIn,,II < 00. If A, c L, x L, is linear for each n 2 I , the extended limit of the sequence {A,} is defined by

(4.3) ex-lim A, = {U; g) c L x L: there exists u,, 8,) E A, for each n-m

n 2 1 such that IIf, - r r J l l + 0 and 11 g, - n,g 11 3 O}.

We leave it to the reader to show that cx-lim,,, A, is necessarily closed in L x L (Problem 11). To see that ex-lim,,,A, need not be single-valued even if each A, is, let

L, = L, a, = I , and A, = B + nC for each n 2 1, where B and C are bounded linear operators on L. If/ belongs to N ( C ) , the null space of C, and h E L, then A,,(f+ (I/n)h)+ Bf+ Ch, so

{(A Bf+ Ch):Je N(C) , h E L} c ex-lim A,. (4.4) n-m

Another situation in which multivalued operators arise is described in the next section.

5. SEMIGROUPS ON FUNCTION SPACES

In this section we want to extend the notion of the generator of a semigroup, but to do so we need to be able to integrate functions u: [O, a)+ L that are

5. SEMICROUIS ON FUNCllON SPACES 23

not continuous and to which the Riemann integral of Section 1 does not apply. For our purposes, the most efficient way to get around this difficulty is to restrict the class of Banach spaces L under consideration. We therefore assume in this section that L is a “function space” that arises in the following way.

Let (M, a) be a measurable space, let r be a collection of positive measures on A, and let 2‘ be the vector space of .,#-measurable functions f such that

(5.1)

Note that 1 1 . [I is a seminorm on Y but need not be a norm. Let N = { f ~ 9’: l l f l l = 0) and let L be the quotient space 9 / N , that is, L is the space of equivalence classes of functions in 9, wheref- g if I[/- gll = 0. As is typically the case in discussions of Lp-spaces, we do not distinguish between a function in Y and its equivalence class in L unless necessary.

L is a Banach space, the completeness following as for E-spaces. In fact, if v is a o-finite measure on A’, 1 s q 5 ao, p - ’ + q - ’ = 1, and

IlSIl --= SUP If1 dP < m. r c r I

(5.2)

where (1 . 11, is the norm on U ( v ) , then L = E ( v ) . Of course, if r is the set of probability measures on A, then L = B(M, A), the space of bounded 4- measurable functions on M with the sup norm.

Let (S, 9, v ) be a a-finite measure space, let f: S x M -+ R be 9’ x A- measurable, and let g: S + 10, 00) be 9’-measurable. If Ilf(s, .)[I 5 g(s) for all s E S and g(s)v(ds) < m, then

(5.3)

and we can define j f ( s , . ) v ( d s ) E L to be the equivalence class of functions in 2’ equivalent to h, where

(5.4)

With the above in mind, we say that u : S-+ L is measurable if there exists an Y x A-measurable function u such that u(s, . ) E u(s) for each s E S. We define a semigroup (T(t)} on t to be measurable if T( * )J is measurable as a function on ([O, m), a[O, 00)) for each/€ L. We define thefull generaror A’ of a measurable contraction semigroup (T(r)} on L by

We note that A is not, in general, single-valued. For example, if L = B(R) with the sup norm and T(t)f(x) s f ( x + t), then (0, g) E A for each y E B(R) that is zero almost everywhere with respect to Lebesgue measure.

5.1 Proposition Let L be as above, and let { T(r)} be a measurable contraction semigroup on L. Then the full generator A of { T(t)) is linear and dissipative and satisfies

for all h E W(A - A) and A > 0. If

T(s) e-"T(t)h dt = I" e-"T(s + t)h dt 0

(5.7)

for all h E L, 1 > 0, and s 2 0, then 5#(1 - 2) = L for every 1 > 0.

Proof. Let V; g) E A, A=- 0, and h = y- g. Then

(5.8) lm e-"T(r)h dr = A dp e-"T(r)fdt - e-"'T(t)g dr

= 1 r e-"T(t)fdt - 1 e-" T(s)g ds dt

= J

Consequently, I l f l I s A - '11 h 11, proving dissipativity, and (5.6) holds.

g = 4.j- h. Then

(5.9) T(s)g ds = 1

Assuming (5.7), let h E L and A > 0, and define f- e-"T(t)h dt and

lm e-'"T(s + u)h du ds - T(s)h ds

= I en* im e-"T(u)h du ds - T(s)h ds

= el'

SI l

e-'"T(u)h du - 1." e-AuT(u)h du

+ T(s)h ds - T(s)h ds

= Wf-f for all t 2 0,soU; g) E Aand h = Af-g E SI (A - A). 0

5. SEMKROUrJONFUNCllONWACES 25

The following proposition, which is analogous to Proposition I.s(a), gives a useful description of some elements of 2.

5.2 Proposition Let L and (T(t)) be as in Theorem 5.1, let h B t and u 2 0, and suppose that

(5.10)

for all I z 0. Then

(5.1 1)

T(t) l T(s)h ds = 1 T(t + s)h ds

(l T(s)h ds, T(u)h - h E A’. ) p d . Put 1 = Zt; T(s)h ds. Then

= I”‘ T(s)h ds - 1 T(SP ds

= 6‘ T(s)(T(u)h - h) ds

for all r 2 0. 0

In the present context, given a dissipative closed linear operator A c L x L, it may be possible to find measurable functions u: KO, a)-+ L and u: [O, oo)+ t such that (u(t), u(t)) E A for every t > 0 and

(5.13) u(t) = u(0) + 4s) ds, t ;I 0. l One would expect u to be continuous, and since A is closed and linear, it is reasonable to expect that

for all t > 0. With these considerations in mind, we have the following multivalued extension of Proposition 2.10. Note that this result is in fact valid for arbitrary L.

26 O I f l A l O I S E M K i R O U r S

5.3 Proposition Let A c L x L be a dissipative closed linear operator. Suppose u : [O, a)-, L is continuous and (so u(s) ds, u(t) - u(0)) E A for each t > 0. Then

(5.15)

for all t 2 0. Given I > 0, define

II u ( 4 II s II 40) II

(5.16) l= e-&u(t) dt , g = 1 e-*"(u(t) - 40)) dr.

Then cf, g ) E A and y- g = u(0).

Proof. Fix r 2 0, and for each E > 0, put u,(t) = ti-'

(5.1 7)

Since (u,(r), & - I ( & + e) - ~ ( 1 ) ) ) E A, it follows as in Proposition 2.10 that IIu,(t)II S llu8(0)ll. Letting&-+ 0, we obtain (5.15).

(5.18) j = e- * 'q t ) dt = 1 e-*l$' u(s) ds dt,

so U; 8) E A by the continuity of u and the fact that A is closed and linear. The equation 1f - g = u(0) follows immediately from the definition offand g. 0

u(s) ds. Then

u,(t) = ~'(0) + E-'(u(s + E ) - u(s)) ds.

Integrating by parts,

Heuristically, if {S(r)} has generator 8 and {T(t)} has generator A + B, then (cf. Lemma 6.2)

(5.19)

for all t 2 0. Consequently, a weak form of the equation u, = (A + B)u is

(5.20)

We extend Proposition 5.3 to this setting.

T(t ) f= S(t)f+ r S ( r - s)AT(s)/ds 0

u(t) = S(t)u(O) + 5' S(t - s)Au(s) ds. 0

5.4 Proposition Let L be as in Proposition 5.1, let A c L x L be a dissipative closed linear operator, and let {S(t)} be a strongly continuous, measurable, contraction semigroup on L. Suppose u: [O, 00)- L is continuous, u: LO, 00)- L is bounded and measurable, and

5. SEMICROWS ON FUNCnON SPAACES 27

(5.21)

for all r z 0. If

(5.22)

for every t > 0, and

(5.23)

for all q. r, r 2 0, then (5.15) holds for all I z 0.

S(q + r)D(s) ds = S(q) S(r)o(s) ds c 5.5 Remark The above result holds in an arbitrary Banach space under the assumption that u is strongly measurable, that is, u can be uniformly approx-

0 imated by measurable simple functions.

Proof. Assume first that u: [O, m)-+ L is continuously differentiable, u : [O, a)--+ L is continuous, and (u(t), 41)) E A for all t z 0. Let 0 = to < t , <

(5.24)

< t , = t. Then, as in the proof of Proposition 2.10,

n

II u(t) I1 = II 40) II + 1 c I1 4tO I1 - II 44 - I ) I l l

28 O?ERATORJEMK;ROU‘S

where s’ = t ,- I and s” = t , for r , - I s s < r , . Since the integrand on the right is bounded and tends to zero as max ( t , - t i , 0, we obtain (5.15) in this case.

In the general case, fix t 2 0, and for each E > 0, put

u(s) ds, u,(t) = e - I“’

l sb+‘ = & - I 1 S(r + S)U(O) ds + & - I

(5.25)

Then

(5.26) u,(t) = u(r + s) ds

U#) = e - I

S(t + s - r)dr) dr ds

= ~ - l S ( t ) (d S(s)u(O) ds + 6- I s’ 5’ S(t + s - r)u(r) dr ds 0 0

+ I’ r S ( t - r)u(r + s) dr ds 0 0

1 = S(t)[.s-I S(s)u(O) ds + 6 - l 5’ I’ S(s - r)u(r) dr ds 0 0

+ S(t - r)ua(r) dr.

By the special case already treated,

(5.27) II u,(t) I1 S )I e - and letting E--, 0, we obtain (5.15) in general. 0

6. APPROXIMATION THEOREMS

In this section, we adopt the following conventions. For n = 1, 2, . . . , L,, in addition to L, is a Banach space (with norm also denoted by I[ 6 11) and n,: L+ L,, is a bounded linear transformation. We assume that sup,, II n, II < 00. We writef.-+fiff. E t,, for each n 2 1,Je L, and lirn,-= [If, - a, I l l = 0.

6.1 Theorem For n = I , 2,. , . , let (T,(t)) and { T(r)) be strongly continuous contraction semigroups on L, and L with generators A, and A. Let D be a core for A. Then the following are equivalent:

(3 intervals.

For each 1 E L, T,(t)n, f -+ T(r)f for all t 2 0, uniformly on bounded

6. APWOXIMATION THEOREMS 29

(b) (c)

For each f E L, T,(l)n,J+ T(t)ffor all t 2 0. For each f~ D, there exists 1, E Q(A,) for each n 2 I such that

j,,-.Jand A,f,--+ Af(i.e., {(J A S ) : / € D ) c e~-Iim,,+~A,,).

The proof of this result depends on the following two lemmas, the first of which generalizes Lemma 2.5.

6.2 Lemma Fix a positive integer n. Let {S,(r)} and {S(t)} be strongly continuous contraction semigroups on L, and L with generators B,, and B. Let

/E 9(B) and assume that n , , S ( s ) j ~ g(B, , ) for all s 2 0 and that B,n,S( * )j: [O, 00) -+ L,, is continuous. Then, for each t 2 0,

(6.1 )

and therefore

(6.2)

S,(t)n, f - n,, S ( f ) j = S,,(C - sWB, n,, - n, B)S(s) fds , L II Sn(t)n, f - n, S ( t V I t 5 II (B , n n - n, B)s(s)/ II ds.

Plod. It suffices to note that the integrand in (6.1) is -(d/ds)S,(t - s)n,S(s)/ for 0 s s ,< t . 0

6.3 Lemma Suppose that the hypotheses of Theorem 6.1 are satisfied together with condition (c) of that theorem. For n = 1, 2,. . . and R > 0, let At and A' be the Yosida approximations of A, and A (cf. Lemma 2.4). Then A: n, f-+ Ayfor everyfe L and R > 0.

Proof. Fix R > 0. Let /E D and g = ( A - A)f By assumption, there exists 1; E B(A,) for each n 2 I such that /;--+fand Ad,-+ AJ and therefore ( A - A,)S, -+ g. Now observe that

(6.3) I1 A:nng- nnA"gl1

= I I [ A Z ( R - A J - 1 - R f ] n , g - n,[RZ(R - A ) - ' - Af-JgII

= A 2 ( 1 ( R - An)- ' ring - nn(A - A)- 'eI t

s R211(R - A n F 1 ring -Lit + R'ItSn - nn(R - A)-'gII

5 LIInng - ( A - An)/nII + nZII/n - nSII for every n 2 I. Consequently, 11 A: n, g - R, A'g II -+ 0 for all g E - At,,). But &(A - AID) is dense in L and the linear transformations Ain,, - n,AL, n = I , 2 , . . . , are uniformly bounded, so the conclusion of the lemma follows.

0 Proof of Theorem 6.1. (a * b) Immediate.

30 OPERATOR SEMICROWS

(b =5 c) Let 1 > 0. f E 4W), and g = (A - A)A so that f = e-"'T(t)g dt. For each n 2 1, put fn = jz e-"X(t)n,,g dr E B(A,). By (b) and the dominated convergence theorem, S,-.l; so since ( A - An)f, = n,g-+ g = (A - A)J we also have A,,&-, A/:

(c =.a) For n = 1, 2,. . . and A > 0, let {Ti(t)} and {T'(r)) be the strongly continuous contraction semigroups on t, and L generated by the Yosida approximations A: and A'. Given/€ D, choose {jJ as in (c). Then

(6.4) T,(l)nn f- nm T(tlf= UtKnn f-L) + CUt) f , - T$l)LI + Ti(tMS, .- n, n+ "CWn f - n, T A W ]

+ nnCT?t).f- T(l)fJ for every n 2 I and t 2 0. Fix to 2 0. By Proposition 2.7 and Lemma 6.3,

lim SUP 11 X(t).t, - T,"(t)LII 5 lim to 11 An S, - Aijn 11 n- w 0 s I sfo n-m

(6.5)

lim t o { II An S. - nn MII + II nn(AS- AWII n - m

+ IInnAY- Afnnf I I + I I A ~ ~ ~ . ~ - L ) I I I s K~oI lA f - AYII,

where A' = sup,((It,((. Using Lemmas 6.2, 6.3, and the dominated convergence theorem, we obtain

(6.6) lim sup 11 T;(t)n, f - n, Ta(r)fII n-m OLILIO

s lim II(R."n. - n,A")T"s)Jl/ ds = 0. n-m

Applying (6.5), (6.6). and Proposition 2.7 to (6.4), we find that

(6.7) SUP I1 T,(t)nnf - n, T(t)fll S 2Kr011A!f- AfII. I - C O O s r s t o

Since I was arbitrary, Lemma 2.4(c) shows that the left side of (6.7) is zero. But this is valid for allfe D, and since D is dense in L, i t holds for allJe L.

0

There is a discrete-parameter analogue of Theorem 6.1, the proof of which depends on the following lemma.

6.4 lemma Let B be a linear contraction on L. Then

(6.8) II BY- en(8-'Yll 5 J;; II BJ-JII

for allfs L and n = 0, 1,. . ..

6. APFUOXIMATION THEOREMS 31

Proof. Fix/€ L and n 2 0. Fork = 0, I , . . . , (6.9)

Therefore

(6.10)

(Note that the last equality follows from the fact that a Poisson random 0 variable with parameter n has mean n and variance n.)

6.5 Theorem For n = I , 2,. . . , let T,, be a linear contraction on L,, let E, be a positive number, and put A, = E;'(T,, - I). Assume that Iim,,,&, = 0. Let { T(t ) } be a strongly continuous contraction semigroup on L with generator A, and let D be a core for A. Then the following are equivalent:

(a) intervals.

(b)

(c)

For each/€ L, T!,!'Cnln,/-t T(t)ffor all t 2 0, uniformly on bounded

For each/€ L, T!,!%,, f- T(t)/for all t 2 0.

For each / E D, there exists S. E L, for each n 2 I such that h4/ and Anf,-+ AJ(i.e., ((J A ~ ) : / E D} c ex-limn.,, A,).

Proof. (a b) Immediate.

(b 3 C ) Let A > 0, / E B(A), and g = (A - AM; so that f = jg e-"'f(t)e dt. For each n 2 I , put

(6.1 I )

32 OPERATORSMCROUIS

By (b) and the dominated convergence theorem,L-+J and a simple calculation shows that

(6.12) (1 - AalL = nag -t rlE,nag a3

+ - 1 + e-Aca) e-A*cnT~+'n,g k = O

for every n 2 1, so ( A - A,).& -, g = ( A - A ) j It follows that A,, S, -+ Af:

(6.13) T!IbJn,J- n, T(r)f

(c * a) Givenfe 0, choose {fa} as in (c). Then

and by Theorem 6.1,

(6.15)

Consequently,

(6.16) lim sup 11 T~l'aln,J- n, T(r)f11 = 0.

But this is valid for all f E D, and since D is dense in L, it holds for all f E L.

lim sup I( exp {&a[ i ] ~ a } n a 1- na VIUII = 0. a - m O S C S I O

n - m 051510

0

6.6 Corollary Let {V ( t ) : f 2 0 ) be a family of linear contractions on L with V(0) = I, and let {T(r)} be a strongly continuous contraction semigroup on L with generator A. Let D be a core for A. If lims40 ~ - * [ V ( & ) f - f j = Affor every/€ D, then, for eachfe L, V(r/n)y-+ T(t)ffor all r r: 0, uniformly on bounded intervals.

Proof. It sunices to show that if { t n ) is a sequence of positive numbers such that in-* r 2 0, then V(t,,/n)"'+ T(t)ffor everyfe t. But this is an immediate consequence of Theorem 6.5 with T. = V(tJn) and E, = tJn for each n 2 I . 0

6. APPROXlMATltM THEOREMS 33

6.7 Codary Let { T(t)), (S(t)}, and (V(r)} be strongly continuous contraction semigroups on L with generators A, B, and C, respectively. Let D be a core for A, and assume that D c 9 ( B ) n 9(C) and that A = B + C on D. Then, for each/ E L.

(6. I 7)

for all r 2 0, uniformly on bounded intervals. Alternatively, if ( E , } is a sequence of positive numbers tending to zero, then, for each/€ L,

(6.18)

for all t 2 0, uniformly on bounded intervals.

Proof. The first result follows easily from Corollary 6.6 with V ( t ) I S(c)U(t) 0 for all t 2 0. The second follows directly from Theorem 6.5.

6.8 Corollary Let (T(t)} be a strongly continuous contraction semigroup on L with generator A. Then, for each / E L, (I - (r/n)A)-"J- T(t)f for all I 2 0, uniformly on bounded intervals. Alternatively, if { e n } is a sequence of positive numbers tending to zero, then, for each f e t, (I - E,,A)-~"'~Y--+ T(t)J for all t ;r 0, uniformly on bounded intervals.

Proof. The first result is a consequence of Corollary 6.6. Simply take V ( i ) = (I - tA ) - ' for each f 2 0, and note that if E > 0 and 1 = E - ' , then

where AI is the Yosida approximation of A (cf. Lemma 2.4). The second result 0 follows from (6.19) and Theorem 6.5.

We would now like to generalize Theorem 6.1 in two ways. First, we would like to be able to use some extension A, of the generator A, in verifying the conditions for convergence. That is, given U; g) E A, it may be possible to find u,, g,) E A, for each n 2 1 such that /. -/ and g,+ g when it is not possible (or at least more diflicult) to find u,, g,) E A, for each n 2 1. Second, we would like to consider notions of convergence other than norm convergence. For example, convergence of bounded sequences of functions pointwise or uniformly on compact sets may be more appropriate than uniform convergence for some applications. An analogous generalization of Theorem 6.5 is also given.

34 N TORS EM CROUPS

Let LIM denote a notion of convergence of certain sequences f, E L,, n = 1,2,. . . , to elementsf€ L satisfying the following conditions:

(6.20) LIMf, = f and LIM g, = g imply

LIM (aJ; + Pg,) = cf+ /?g for all a, /3 E R.

(6.21) LIMf:) = f k ) for each k 2 1 and lim sup ll/!hJ -J, 11 V llj4kJ -/[I = 0 imply LIMA, =/:

There exists K > 0 such that for eachfe L, there is a sequenceA, E L, with Ilf.11 s KIIfII, n = 1, 2,. . . , satisfying L I M L =f.

h - m r Z 1 , (6.22)

If A, c L, x L, is linear for each n 2 1, then, by analogy with (4.3). we define

(6.23) ex-LIM A, = (U; g) E L x L: there exists ( f . , 8,) E A,

for each n 2 1 such that LIMA, =/and LIM g, = g}.

6.9 Theorem For n = 1, 2,. . . , let A, c L, x L, and A c L x L be linear and dissipative with 9 ( A - A,) = L, and 9 ( A - A) = L for some (hence all) A > 0, and let {T , ( r ) } and {T(t)) be the - corresponding strongly continuous contraction semigroups on 9 ( A , ) and 9 ( A ) . Let LIM satisfy (6.20H6.22) together with

(6.24) LIMf, = 0 implies LIM (A - A,)-% = 0 for all 1 > 0.

(1) If A c ex-LIM A,, then, for each U; g) E A, there exists u,, 9,) E A, for each n z 1 such that sup, /If. 11 < 00, sup, II g, II < 00, LIM J , =f, LIM 8, = g, and LIM T,(t)J, = T(r)ffor all t 2 0.

(b) If in addition {x(r)} extends to a contraction semigroup (also denoted by { x(t)}) on L, for each n 2 1, and if

(6.25) LIMA = 0 implies LIM T,(r)f. = 0 for all t 2 0,

then, for eachfe B(A), LIMJ; =/implies LIM x(t)f. = T(t)/for all t 2 0. -

6.10 Remark Under the hypotheses of the theorem, ex-LIM A, is closed in L x L (Problem 16). Consequently, the conclusion of (a) is valid for all UI Q) E A’. 0

Proof. By renorming L,, n = 1, 2, ..., if necessary, we can assume K = 1 in (6.22).

Let 2’ denote the Banach spa& (naLILJ x L with norm given by I I ( { L J s f)II = SUPnz 1111; II V IIf II, and let

(6.26)

6. APFROXlMATlON THKMFMS 35

Conditions (6.20) and (6.21) imply that Yo is a closed subspacc of 9, and Condition (6.22) (with K = 1) implies that, for each/€ L, there is an element ( { f n } , / ) 6 9 0 with II({fn}*AII = IIJll.

Let

(6.27) d = {[({fn}*jh ({gn}. 911 E 9 X 9: Un. gn) An for each

n 2 1 and U; g)E A } .

Then I is linear and dissipative, and @(A - .d) = Y for all 1 > 0. The corresponding strongly continuous semigroup {.T(f)} on 9(d) is given by

-

(6.28)

We would like to show that

(6.29)

To do so, we need the following observation. If V; g) E A, 1 > 0, h = AJ- g, ((hn), h ) E Y o . and

(6.30) (f" * 9,) = ((A - A n ) * ' k 9 - h n )

for each n z I , then

To prove this, since A c ex-LIM A,,, choose c/"., 8,) E A, for each n 2 1 such that LIM3, = f and LIM 3, = g. Then LIM (h, - (ly", - 8,)) = 0, so by (6.24), LIM (1 - A,)- 'h, -f, = 0. It follows that LIMf, = LIM ( A - A,,)-*h, = LIMA =f and LIM g,, = LIM (@, - h,) = V- h = g. Also, sup, I I j , II s 1- I SUP, II h, 11 < 00 and SUP. II gn II 5 2 SUP, II h n 11 -= 00. Consequently, [({h), n, ((9,). g)] belongs to 9, x Yo, and it clearly also belongs to d.

Given ( {h , } , h) E Yo and rl > 0, there exists c(, g) E A such that ly- g = h. Define u,, g,) E A, for each n z 1 by (6.30). Then (A - d ) - ' ( { h , , } , h) = ( { f n } , J ) E 90 by (6.31)v SO

(6.32) (1 - d)- ' : 9 0 3 Y o , L > 0.

By Corollary 2.8, this proves (6.29). To prove (a), let (1 g) E A, A > 0, and h = Af- g. By (6.22). there exists

( { h , } , h ) E Yo with ll({h,,}, h) II = IIh 11. Define (h, g,) E A, for each n 2 1 by (6.30). By (6.31). (6.29), and (6.28), ({T,,(t)f,,}, T(t)f) E Yo for all t 2 0, so the conclusion of (a) is satisfied.

As for (b), observe that, by (a) together with I_ (6.25), L I M L = f B B(A) implies LIM T(t)/, -- T(t)ffor all t 2 0. Letfs d ( A ) and choose {$&I} c B(A) such that II /''I -/[I s 2-& for each k 2 1. Put Po' = 0, and by (6.22), choose

for each k 2 1. Since

(6.34)

and

for each n 2 1 and k 2 1, (6.21) implies that

(6.36) Q, m

LIM 1 u!'~ ==A LZM T,(t) C ut) = T(t)J; I I

so the conclusion of (b) follows from (6.25). 0

6.11 Theorem For n = 1, 2,..., let T, be a linear contraction on L,, let E, > 0, and put A, = &;'(T, - I). Assume that limn-m c,, = 0. Let A c L x L bc linear and dissipative with 9?(1 - A) = L for some (hence all) 1 > 0, and let IT(t)} be the corresponding strongly continuous contraction semigroup on 9 ( A ) . Let LIM satisfy (6.20)-(6.22), (6.24), and

(6.37) lim Jjh II = 0 implies LIM 2= 0.

W If A c ex-LIM A,, then, for each U; g) E A, there exists f,, E L, for each n 2 1 such that sup,Ilf,jl < 00, sup,)I A,J,(I < 00, LIMA -I; LIM AJ, = g, and LIM c'h!& = T(r)/for all r z 0.

(6.38) LIMJ, = 0 implies LIM T!/'-y, = 0 for all t 2 0,

then for eachftz 9 ( A ) , LIMA Efimplies LIM c/'"!f,, = T(t)ffor all r 2 0.

(bJ If in addition

-

Proof. Let U; g) E A. By Theorem 6.9, there cxistsI; E L, for each n L 1 such that SUp,!lfn I1 < a, sup,II Af,Il < 00, LIMf, -S, LIM AS, = g, and LJM e'"X = T(r)Jfor all t 2 0. Since

(6.39)

7. NRTUROATION THEOREMS 37

for all t 2 0, we deduce from (6.37) that

(6.40)

The conclusion of(a) therefore follows from (6.14) and (6.37). The proof of (b) is completely analogous to that of Theorem 6.9 (b). 0

7. PERTURBATION THEOREMS

One of the main results of this section concerns the approximation of semigroups with generators of the form A + B, where A and B themselves generate semigroups. (By definition, O(A + B) = O(A) n 9(B).) First, however, we give some suflicient conditions for A + B to generate a semigroup.

7.1 Theorem Let A be a linear operator on L such that A’ is single-valued and generates a strongly continuous contraction semigroup on L. Let B be a dissipative linear operator on L such that 9 ( B ) 3 9 ( A ) . (In particular, 6 is single-valued by Lemma 4.2.) If

where 0 5 a c I and /I 2 0, then A + B is single-valued and generates a strongly continuous contraction semigroup on L. Moreover, A + B = A + 8.

Proof. Let y 2 0 be arbitrary. Clearly, 9 ( A + yB) = 9 ( A ) is dense in L. In addition, A + yB is dissipative. To see this, let A be the Yosida approximation of A’ for each p > 0, so that A, = p[p(p - .$)-I - 11. If /€ d ( A ) and A > 0. then

by Lemma 24c) and the dissipativity of yB.

I f j e 9(A), then there exists (f.} c 9 ( A ) such thatf.+/and AS,-+ 26 BY (7.1), {Bf;) is Cauchy, s o f ~ 9 ( B ) and BS,+ BJ Hence 9 ( J ) t 9(B) and (7.1) extends to

(7.3) In addition, i f /€ 9(A) and if (I,) is as above, then

(7.4)

implying that A -t- yB is a dissipative extension of A' + ys.

(7.5) T = { y 2 0: 4?(6 - A' - yb) = L for some (hence all) 6 > 0).

To complete the proof, it suffices by Theorem 2.6 and Proposition 4.1 to show that 1 E r. Noting that 0 E r by assumption, it is enough to show that

( A + yg)f= lim A& + y lim Bf. = lim ( A + yB)/, = ( A + yE)J a a a

Let

1 - ay y E r n Lo, 1) implies [y, y -+ 7) c r

To prove (7.6), let y E r n [O, I), 0 5 E < (2a)-'(l - ay), and L > 0. If g E B(A), then/= (I - A - y@- ' g satisfies

(7.7)

by (7.3), that is,

(7.8)

and consequently,

(7.9) IIB(L-A-; .B)- 'g l i ~ [ 2 a ( l -q)-'+/?(~ - a y ) - ' ~ - ' ] l l g l l .

Thus, for I suficiently large, I IE&(A - A - B)-'II < 1 , which implies ,that I -

11 lgsrr 5: all 4.31 + 811f11 dl(A + rb)fll + aril mr + Plifl i

Ilj3Jll 5 - aY)-'JJ(A' + ytr>/n + P(1 - aY)-'llJIl,

- A' - yb)-' is invertible. We conclude that

(7.10) B(6 - A' - ( y -k e)B) 3 .@((A - A - ( y 4- 6)&1 - A - yB)-') = @ ( I - &&I - A'- yB)-') = L

for such 6, so y + E E r, implying (7.6) and completing the proof. 0

7.2 Corollary If A generates a strongly continuous contraction semigroup on L and E is a bounded linear operator on L, then A + B generates a strongly continuous semigroup { T(t)) on L such that (7.1 I ) 11 T(r)i) 5 e"'"', r 2 0.

Proof. Apply Theorem 7.1 with B - [I B 11 I in place of B. El

Before turning to limit theorems, we state the following lemma, the proof of which is left to the reader (Problem 18). For an operator A, let M ( A ) 5 {fe .$@(A): Af = 0 ) denote the null space of A.

7.3 Lemma Let B generate a strongly continuous contraction semigroup {S(t)) on L, and assume that

(7.12) tim A e-"S(r)(dr = Pf exists for all (e L.

Then the following conclusions hold :

a - o +

(a) P is a linear contraction on L and P2 = P. (b) S(r)P = PS(r) = P for all t 2 0. (c) @ P ) = XCB). - (d) N( P ) = W( E).

7.4 Remark If in the lemma

(7.13) B = y - ' ( Q - I ) ,

where Q is a linear contraction on L and y > 0, then a simple calculation shows that (7.12) is equivalent to

m

(7.14) lim ( I - p) 1 pkQL/= Pf exists for all /E L. 0 p - l - k = O

7.5 Remark holds and

If in the lemma lim,+m S(r)( exists for every /E L, then (7.12)

(7.15) Pf = lim S(i)J / E L. t -m

If E is as in Remark 7.4 and if limk-m QY exists for every (E L. then (7.14) holds (in fact, so does (7.15)) and

(7.16) Pf= lim Q? (E L. k-m

The proofs of these assertions are elementary. 0

For the following result, recall the notation introduced in the first paragraph of Section 6, as well as the notion of the extended limit of a sequence of operators (Section 4).

7.6 Theonm Let A c L x L be linear, and let B generate a strongly continuous contraction semigroup {S(t)} on L satisfying (7.12). Let D be a subspace

40 OPERATORS€MIGROWS

of 9 ( A ) and D' a core for B. For n = 1,2,. . . , let A, be a linear operator on L, and let a, > 0. Suppose that limn,man = 00 and that

(7.17) {U; g) E A : f E D} c ex-lim A,, n - e

(7.18) ((h, Bh): h E D') t ex-Jim a;'A,.

Define C = (U; fg): U; g) E A, f~ D} and assume that {(A g) E c: g E 0) is single-valued and generates a strongly continuous contraction semigroup { ~ ( c ) } on 6.

n-oD

(a) If A, is the generator of a strongly continuous contraction semigroup {F( t ) } on La for each n 2 1, then, for eachfe 6, x(t)nJ--r T(t)ffor all 2 0, uniformly on bounded intervals.

(b) If A , = E,-I(T, - I) for each n 2 1, where T. is a linear contraction on L, and E, > 0, and if lim,,,~, = 0, then, for each f E D, T!'%, f-. T(f)f for all f 2 0, uniformly on bounded intervals.

Proof. Theorems 6.1 and 6.5 are applicable, provided we can show that

(U; g) E C: g E 6) c ex-Jim A, n (b x 6). (7.19) ( n - r n ) Since ex-lim,,, A, is closed, it suffices to show that C c ex-limn,, A,. Given U; g) B A with f tz D, choose f . E 9 ( A n ) for each n 2 1 such that fa- f and A,f , - , g. Given h E D', choose h, E B(A,) for each n 2 I such that h,+ h and a,- ' A , h, + Bh. Then f . + a, 'h , -+ f and A,cf, + a; 'h,) 3 g + Bh. Conse- quently,

(7.20) {U; g + Bh): U; g) E A , f E D, h E D'} c ex-lim A,.

But since ex-limn,, A, is closed and since, by Lemma 7.3(d),

(7.21)

for all g E L, we conclude that

(7.22)

1-4)

7 -

Pg - g E M ( P ) = 9 ( B ) = 9t(B(n*)

{U; Pg): V; g) E A , f e D ) c ex-lim A,, n-m

completing the proof. 0

We conclude this section with two corollaries. The first one extends the conclusions of Theorem 7.6, and the other describes an important special case of the theorem.

7.7 Corollary Assume the hypotheses of Theorem 7.qa) and suppose that (7.15) holds. If h E M(P) and if { t , } c 10, GO) satisfies tima,, t.u, = 00,

7. PERTUIIATION THEOREMS 41

then T,,(r,)n,h-+ 0. Consequently, for each f E P - ' ( 6 ) and 6 E (0, I), %(r)n,f-+ T(r)P/; uniformly in b s t g 6 - ' .

Assume the hypotheses of Theorem 7.6(b), and suppose that either (i) lim,,,a,q, = 0 and (7.15) holds, or (ii) lim,,-.,a,,c, = y > 0 and (7.16) holds (where Q is as in (7.13)). If h E N(P) and if {&,) c (0, 1,. . .} satisfies

k,a, E, = m, then TFn, h -+ 0. Consequently, for each f E P - '(6) and 6 E (0, I), T!"%J-, T(~)PJ uniformly in b s r 5 6 - '. Proof. We give the proof assuming the hypotheses of Theorem 7.6(a), the other case being similar. Let b E J(r(P), let (t,} be as above, and let E > 0. Choose s 2 0 such that II S(s)h II 5 c / 2 K , where K = supnr I 11 n, 11, and let s, = sAr,a, for each n 2 I . Then

for all n suficiently large by (7.18) and Theorem 6.1. If J E L, then f - Pf E .N(P), so 7Jrn)n,(J- Pf)+ 0 whenever { t , } c LO, 00) satisfies t, = r # 0. If f e P-'(d), this, together with the conclusion of the theorem

0 applied to PJ completes the proof.

7.8 Corollary Let l l ,A , and B be linear operators on L such that B generates a strongly continuous contraction semigroup {S(r)) on L satisfying (7.12). Assume that 9(n) n 9 ( A ) n 9 ( E ) is a core for B. For each a sufkiently large, suppose that an extension of ll + aA + a'E generates a strongly continuous contraction semigroup { T , ( r ) } on L. Let D be a subspace of

(7.24) ( / E 9(n) n 9 ( A ) n .N(B):

there exists h E Q(n) n 9 ( A ) n 9(B) with Bh = - A / } ,

and define

Then C is dissipative, and if ((J 8) E c: g E 0). which is therefore single- valued, generates a strongly continuous contraction semigroup (T(r ) ) on 6, then, for eachJE D, lima+,., x(r)/= T(r)/for all t 2 0, uniformly on bounded intervals.

Proof. limn+m a, = GO, and apply Theorem 7.qa) with L, = L, n, = I, A replaced by

(7.26) (U; n/+ A h ) : / € D, h E 9(n) n 9 ( A ) n 9 ( B ) , Bh = -A!},

A, equal to the generator of {T*(r)}, a, replaced by af. and D = WJ) n 9 ( A ) n 9(B). Since A,,cf+ a;'h) = nf+ Ah + a i ' l l h whenever/€ D, h E 9(n) n 9 ( A ) n 9(B), Bh = -AS, and n 2 1, and since limn--

Let {a,} be a sequence of (sufficiently large) positive numbers such that

42 OIEIATORSMGROUIS

a,-2A,h = Bh for all h E D', we find that (7.17) and (7.18) hold, so the theorem is applicable. The dissipativity of C follows from the dissipativity of ex-lim,,, A". 0

7.9 Remark (a) Observe that in Corollary 7.8 it is necessary that PAf= 0

(b) Let /E 9 ( A ) satisfy PAf= 0. To actually solve the equation for a l l f E D by Lemma 7.3(d).

Bh = - Affor h, suppose that

(7.27) II(s(t) - p)g 11 dt < 00, g E L.

Then h -" limA-o+(A - B)-'Af= j; (S(t) - P)A/dt belongs to 9(B) (since B is closed) and satisfies Bh = -A$ Of course, the requirement that h belong to 9(n) A 9 ( A ) must also be satisfied.

(c) When applying Corollary 7.8, it is not necessary to determine C explicitly. instead, suppose a linear operator Co on b can be found such that Co generates a strongly continuous contraction semigroup on b and Co c C. Then {V; g) E (f: g E b} = Co by Proposition 4.1.

(d) See Problem 20 for a generalization and Problem 22 for a closely related result. 0

8.

1.

2.

3.

4.

PROBLEMS

Define { T(r)} on &R) by T(t)J(x) =/(x + I). Show that { T(t)} is a strongly continuous contraction semigroup on t, and determine its generator A. (In particular, this requires that 9 ( A ) be characterized.)

Define { T(r)} on c(R) by

for each r > 0 and T(0) = I. Show that {T( t ) } is a strongly continuous contraction semigroup on L, and determine its generator A.

Prove Lemma 1.4.

Let (T(r)} be a strongly continuous contraction semigroup on L with generator A, and let/€ 9 ( A 2 ) . (a) Prove that

Jo

a m o m 43

(b) Show that II ASII' 5 411 A'JII 11/11. Let A generate a strongly continuous semigroup on L. Show that fl.i I

9 ( A " ) is dense in L.

Show directly that the linear operator A = f d 2 / d x z on L satisfies conditions (a)-@) of Theorem 2.6 when 9 ( A ) and L are as follows: (a) g ( ~ ) = { f ~ C2[0, 11: a,f"(i) - (- l)'&f'(i) = 0, i = 0, I } .

L = CCO, 11. ao. Po. a I , PI 2 0, a. I- Po 7 0, al i- P I > 0.

(b) L@(A)-= {fe C'CO, 00): ao/"(0) - Bof'(O) = 0) L = CCO, 001, ao, Po 2 0, a. + Do > 0.

(c) Hint: Look .for solutions of A , - 4 /" = g of the form f ( x ) = exp { - a x } & ) .

Show that CF(R) is a core for the generators of the semigroups of Prob- lems 1 and 2.

In this problem, every statement involving k, I, or n is assumed to hold for all k, I, n 2 1.

be a sequence of closed subspaces of L. Let 0,. M, , and MP' be bounded linear operators on L. Assume that u, and Mp) map L, into L, , and that for some fl, > 0, II MP'II < fi, and

9 ( A ) -- C,(Pa), L = Qua).

Let L , c L, c L, c *

5.

6.

7.

8.

lim I( Mf"" - M, 1) = 0. r))" m

Suppose that the restriction of A, I that there exist nonnegative constants dlk( (= a(,), & I , and y such that

(8.4)

Mf"Uj to L, is dissipative and

f E t, II u h U J - UI UJll s ad11 UJll + IIUJII),

(8.7)

Define A =

(8.8)

If 9 ( A ) is dense in L, show that A is single-valued and generates a strongly continuous contraction semigroup on L.

I Mj [I, on

1 OD W

~ ( A I = {I . u Ln: 1 fijllujflI < 00 . n=l j = J

4 (wMToII6McROWS

Hint: Fix A > 3y and apply Lemma 3.6. Show first that for g E 9 ( A ) and f n * (a - AA-’g,

n

1-1 (8.9) ( a - Y ) I l U d ~ l l IIuhg l l + (fikJ+r,akj)lluj/;ll.

Denoting by p the positive measure on the set of positive integers that gives mass P h to k, observe that the formula

(8.10)

defines a positive bounded linear operator on L’(p) of norm at most 27.

9. As an application of Corollary 3.8, prove the following result, which yields the conclusion of Theorem 7.1 under a different set of hypotheses.

Let A and E generate strongly continuous contraction semigroups { T(r)) and {S(t)} on L. Let D be a dense subspace of L and 111 * 111 a norm on D with respect to which D is a Banach space. Assume that 111fIII 2 11/11 for allfc D. Suppose there exists p 2 0 such that

(8.1 1) D = W2); II A’Ill S rlllflll, f Q D ;

(8.12)

(8.13) T(t): D - , 0, S(t): D-, D, t 2 0;

(8.14) 111 W) 111 s e’, 111 S(0 111 s e”’, 2 0.

Then the closure of the restriction of A + B to D is single-valued and generates a strongly continuous contraction semigroup on L.

We remark that only one of the two conditions (8.11) and (8.12) is really needed. See Ethier (1976).

10. Define the bounded linear operator E on L = C([O, 13 x [O, 11) by Bf(x, y) =

(8.15)

f ( x , z) dz, and define A c L x L by

A = {U t/,= + W:SE C2(C0, 13 x CO, 11) n W ? A fA0, Y) =f3, y) = 0 for all y E LO, 11,

h E Jlr(B)).

Show that A satisfies the conditions of Theorem 4.3.

11. Show that ex-lim,,, A,, defined by (4.3X is closed in L x L. 12. Does the dissipativity of A, for each n 2 1 imply the dissipativity of

ex-lim,,, A,?

13. In Theorem 6.1 (and Theorem 6.5). show that (a)-+) are equivalent to the following:

a. raocmts 45

(d) There exists 1 > 0 such that ( A - A, ) - ’n , ,g+(1 - A ) - ’ g for all g E L.

14. Let L, {L,,}, and In,) be as in Section 6. For each n 2 1, let {T , ( t ) ) be a contraction semigroup on L,, or, for each n 1 I, let ( T , ( r ) } be defined in terms of a linear contraction T, on L, and a number E, > 0 by 7Jr) =

E, = 0. Let { T(t)} be a contraction semigroup on L, let J g E L, and suppose that lim,4m T ( t ) j = 8 and

for all t 2 0; in the latter case assume that

(8.16) lim sup I[ 7Jr)nJ- n, T(r)jII = 0

for every ro > 0. Show that

(8.17)

if and only if

(8.18)

IS. Using the results of Problem 2 and Theorem 6.5, prove the central limit theorem. That is, if X,, X,, . . . are independent, identically distributed, real-valued random variables with mean 0 and variance I , show that n- c;= I X , converges in distribution to a standard normal random variable as n-+ 00. (Define TJ(x) = E u ( x + n - ’ ” X , ) ] and c, = n-’,)

Under the hypotheses of Theorem 6.9, show that ex-LIM A,, is closed in L x L.

17. Show that (6.21) implies (6.37) under the following (very reasonable) additional assumption.

(8.19) If j, E L, for each n 2 1 and if, for some no 2 1,j, = 0 for all n 2 n o , then LIMS, = 0.

Prove Lemma 7.3 and the remarks following it.

Under the assumptions of Corollary 6.7, prove (6.18) using Theorem 7.6. Hinr: For each n 2 I , define the contraction operator T, on L x L by

n-. w 0 SI 610

lim sup 11 T,(t)nJ- nn T(t)fll = 0 n - w 120

lim sup II T,(r)n,g - n, T(r)g )I = 0. n - m t a O

16.

18.

19.

(8.20)

20. Corollary 7.8 has been called a second-order limit theorem. Prove the following kth-order limit theorem as an application of Theorem 7.6.

Let A,,, A , , . . . , A, be linear operators on L such that A, generates a strongly continuous contraction semigroup {S(c)} on L satisfying (7.12). Assume that 5% = n $ - 0 9 ( A , ) is a core for A,. For each a suficiently

16 OPEUTORS€MICIOUPS

large, suppose that an extension of Cf=oajAj generates a strongly continuous contraction semigroup { 7 J f ) ) on L. Let D be a subspace of

(8.21) {fo E 9: there exist f l , f z , . . . , f , - l E .9 with

I m

110 A L - m + j / ; = O for m = O , . . . , k - 1 ,

and define

I k - I

l = O (8.22) C = {(fo, PAj&): fo E D,f,,. . . ,&-, as above .

Then C is dissipative and if {U; g) E c: g E 61, which is therefore single- valued, generates a strongly continuous contraction semigroup { T(r)} on 6, then, for eachfE 6, lima-,,,, 'lf&)f= T(t)f for all t 2 0, uniformly on bounded intervals.

21. Prove the following generalization of Theorem 7.6. Let M be a closed subspace of L, let A t L x L be linear, and let B,

and B , generate strongly continuous contraction semigroups (S , ( t ) } and {S,(r)} on M and L, respectively, satisfying

(8.23) lim R 1 e-A"S,(t)fdr = P , f exists for all ffs M,

(8.24) lim R e-"'S,(f)fdt -= P, f exists for all f E L.

Assume that @P,) c M. Let D be a subspace of 9 ( A ) , D, a core for B , , and D, a core for B,. For n = 1. 2,. . . , let A, be a linear operator on L, and let a,, /In > 0. Suppose that lim,-ma, = 00,

(8.25) (U; g) E A : ~ E D} c ex-lim A,,

(8.26) {(h, B , h ) : h c D,} c ex-lim a;'A,,

(8.27) {(k, B , k): k E D2} c ex-lim 'A,.

Define C = {U; P I P , 9): (J g) E A , f e D } and assume that {U g) e c: g E b} generates a strongly continuous contraction semigroup { ~ ( t ) f on D. Then conclusions (a) and (b) of Theorem 7.6 hold.

22. Prove the following modification of Corollary 7.8. Let n, A. and B be linear operators 0 1 1 L such that 8 generates a

strongiy continuous contraction semigroup {S(C)) on L satisfying (7.12). Assume that 9(n) n D(A) n B(B) is a core for B. For each a sufkiently large, suppose that an extension of ll + aA + a2B generates a strongly

A - O +

A-O+ c /I, = 00, and

n-m

n-m

n-m

9. NOTES 47

continuous contraction semigroup { T#)) on t. Let D be a subspace of 9(n) n 9 ( A ) n N(B) with m P ) c 6, and define C = {(J P A / ) : / E D}. Then C is dissipative. Suppose that c generates a strongly continuous contraction semigroup { V(r)} on D, and that

m

(8.28) lim L [ e-"U(r)fdt = P,f exists for every f e 6. A - O + JO

Let Do be a subspace of {/E D: there exists h E 9(n) n 9 ( A ) n 9 ( B ) with Bh = - A t } , and define

(8.29) Co = {(J P o P n f + P , P A h ) : / € Do,

h E 9(n) n 9 ( A ) n 9(B), Bh = -AS).

Then C, is dissipative, and if {U; 8 ) E co: g E a,} generates a strongly continuous contraction semigroup { T(r)) on 6,. then, for each /E Do, Iirnadm T&)f= T(r)/for all t 2 0, uniformly on bounded intervals.

23. Let A generate a strongly continuous semigroup {T( t ) } on L, let B(t ) : L-4 L, t 2 0, be bounded linear operators such that (B( t ) } is strongly continuous in t L 0 (i.e., t-+ B(r)fis continuous for eachJE L). (a) Show that for each f~ L there exists a unique u : [O , o o ) ~ L

satisfying

(8.30) ~ ( t ) = T(t)f+ T(t - s)B(s)u(s) ds.

(b) Show that if B(t)g is continuously differentiable in c for each g E L, and f E 9 ( A ) , then the solution of (8.30) satisfies

(8.31) a - u(t) = Au(r) + B(t)u(t). at

9. NOTES

Among the best general references on operator semigroups are Hille and Phillips (1957), Dynkin (1965), Davies (1980), Yosida (1980). and Pazy (1983).

Theorem 2.6 is due to Hille (1948) and Yosida (1948). To the best of our knowledge, Proposition 3.3 first appeared in a paper of

Watanabe (1968). Theorem 4.3 is the linear version of a theorem of Crandall and Liggett

(1971). The concept of the extended limit is due to Sova (1967) and Kurtz ( 1 969).

Sufficient conditions for the convergence of semigroups in terms of convergence of their generators were first obtained by Neveu (1958). Skorohod (l958), and Trotter (1958). The necessary and suflicient conditions of Theorems

48 OrUATORSMCIOUPS

6.1 and 6.5 were found by Sova (1967) and Kurtz (1969). The proof given here follows Goldstein (1976). Hasegawa (1964) and Kato (1966) found necessary and sufficient conditions of a different sort. Lemma 6.4 and Corollary 6.6 are due to Chernoff (1968). Corollary 6.7 is known as the Trotter (1959) product formula. Corollary 6.8 can be found in Hille (1948). Theorems 6.9 and 6.1 1 were proved by Kurtz (1970a).

Theorem 7.1 was obtained by Kato (1966) assuming a < and in general by Gustafson (1966). Lemma 7.3 appears in Hille (1948). Theorem 7.6 is due to Ethier and Nagylaki (1980) and Corollary 7.7 to Kurtz (1977). Corollary 7.8 was proved by Kurtz (1973) and Kertz (1974); related results are given in Davies (1 980).

Problem 4(b) is due to Kallman and Rota (1970), Problem 8 to Liggett (1972), Problem 9 to Kurtz (see Ethier (1976)), Problem 13 to Kato (1966), and Problem 14 to Norman (1977). Problem 20 is closely related to a theorem of Kertz (1978).

2

This chapter consists primarily of background material that is needed later. Section I defines various concepts in the theory of stochastic processes, in particular the notion of a stopping time. Section 2 gives a basic introduction to martingale theory including the optional sampling theorem, and local martingales are discussed in Section 3, in particular the existence of the quadratic variation or square bracket process. Section 4 contains additional technical material on processes and conditional expectations, including a Fubini theorem. The DoobMeyer decomposition theorem for submartingales is given in Section 5, and some of the special properties of square integrable martingales are noted in Section 6. The semigroup of conditioned shifts on the space of progressive processes is discussed in Section 7. The optional sampling theorem for martingales indexed by a metric lattice is given in Section 8.

STOCHASTIC PROCESSES AND MARTINGALES

1. STOCHASTIC PROCESSES

A stochastic process X (or simply a process) with index set 1 and state space (E, a) (a measurable space) defined on a probability space (Cl, 9, P) is a function defined on 1 x Q with values in E such that for each r E 1, X(t, .): R-+ E i s an E-valued random variable, that is, { U J : X(f , UJ) E r} E .F for every E a. We assume throughout that E is a metric space with metric r

49



50 STOCHfiTIC PROCESS AND MARTINGALES

and that 1 is the Bore1 a-algebra B(E). As is usually done, we write X ( t ) and X(t , * ) interchangeably.

In this chapter, with the exception of Section 8, we take N = [O, 00). We are primarily interested in viewing X as a “random” function of time. Conse- quently, it is natural to put further restrictions on X. We say that X is measurable if X: [O, 00) x f2-t E is g[O, 00) x $-measurable. We say that X is (almost surely) continuous (right continuous, lefz continuous) if for (almost) every o E R, X( ., w ) is continuous (right continuous, left continuous). Note that the statements “ X is measurable” and “X is continuous” are not parallel in that “X is measurable” is stronger than the statement that X( ., w ) is measurable for each o E R. The function X( -, a) is called the sample path of the process at w.

A collection (S,} E {F,, t E LO, 00)) of 0-algebras of sets in F is a fir- tration if 9, c $,+, for t , s E [O, m). Intuitively 9, corresponds to the information known to an observer at time t . In particular, for a process X we define (4:) by 9; = a(X(s): s 5 c); that is, 9: is the information obtained by observing X up to time t .

We occasionally need additional structure on {9J. We say {S,} is right continuous if for each t L 0, SI =,sit,. = r)a,04tlt,. Note the filtration {F,+} is always right continuous (Problem 7). We say (9,) is complete if (a, 9, P) is complete and { A E 9 : P(A) = 0) c So,

A process X is adapted to a filtration {S,) (or simply {F,}-adapted) if X(r) is 6,-measurable for each t L 0. Since 6, is increasing in I, X is {$,}-adapted if and only if 9; c S, for each t 2 0.

A process X is {.F,}-progressive (or simply progressive if (9,) = (9:)) if for each t 2 0 the restriction of X to [ O , t ] x R is &[O,t] x 9,-measurable. Note that if X is {4F,}-progressive, then X is (FJ-adapted and measurable, but the converse is not necessarily the case (see Section 4 however). However, every righf (left) continuous (9J-adapted process is {.F,}-progressive (Problem 1).

There are a variety of notions of equivalence between two stochastic processes. For 0 s f , < t2 < * - * < f , , let p,, , . . . . ,- be the probability measure on g ( E ) x - . * x 9 ( E ) induced by the mapping ( X ( t , ) , . . . , X(c,))- Em, that is, p I , * . . . , , ~ r ) = P{(X( t , ) , . . . , X(t , ) ) E r}, r E a ( E ) x - - x @(E). The probability measures {p , , , . . , , m 2 1, 0 5 t , < * e . < t,} are called the Jinite- dimensional distributions of X. If X and Y are stochastic processes with the same finite-dimensional distributions, then we say Y is a version of X (and X is a version of Y). Note that X and Y need not be defined on the same probability space. If X and Y are defined on the same probability space and for each c 2 0, P ( X ( t ) = Y(t)} = 1, then we say Y is a modijication of X. (We are implicitly assuming that (X(t) , Y(t)) is an E x E-valued random variable, which is always the case if E is separable.) If Y is a modification of X, then clearly Y is a version of A’. Finally if there exists N E 9 such that ON) = 0 and X( -, w ) = Y( a , w ) for all w $ N, then we say X and Y are indistinguishable. If X and Y are indistinguishable, then clearly Y is a modification of X.

1. STOCHASTIC m o m m SI

A random variable T with values in [O, GO] is an {9,}-stopping time if {I s t } E 9, for every t 2 0. (Note that we allow I = 00.) If I < 00 as., we say I isfinite as. If T s 7' < 00 for some constant T, we say T is bounded. In some sense a stopping time is a random time that is recognizable by an observer whose information at time t is 9,.

If r is an {PI)-stopping time, then for s < r, { T s s} E 9, c 9,, { T < t } = U,(z I; I - l/n} E 9, and (I = t } = {I 5 t } - (z < t } e 9,. If T is discrete (i.e., if there exists a countable set D c [O, 003 such that {I E D) = a), then I is an (9,)-stopping time if and only if {I = t } E S, for each t E D n [O, m).

1.1 Lemma A [O, 001-valued random variable T is an {Pl+)-stopping time if and only if { I < t} E 9, for every t 2 0.

Proof. If { t < t } e 9, for every t z 0, then {I < t + n - I } E St+,-, for n 2 m and { 7 < 11 = on{? < t + n u ' ) E flm91+m-, = .(PI+. The necessity was observed above. 0

1.2 Proposition Then the following hold.

Let t l r T ~ , . . . be {SF,}-stopping times and let c E [O,oo).

(a) r l + c and A c are {9,}-stopping times. (b) sup, I, is an { .F,}-stopping time. (c) minks,. rk is an {9,}-stopping time for each n 2 1. (d) If (9,) is right continuous, then inf,r,, and I,

- are {F,}-stopping times.

Proof. We prove (b) and (d) and leave (a) and (c) to the reader. Note that {sup,, I" s t } = on{ z, s t } E: PI so (b) follows. Similarly {inf,,?, e t ) = U,{I, < I} E PI , so if (9,) is right continuous, then inf,?, is a stopping time by Lemma 1.1. Since iimn4rn T,, = ~up,,,inf,,~,,,~, and limn-* z, = inf,sup,,,r,, (d) follows. 0

-

By Proposition 1.2(a) every stopping time I can be approximated by a sequence of bounded stopping times, that is, limn-m T A n = I. This fact is very useful in proving theorems about stopping times. A second equally useful approximation is the approximation of arbitrary stopping times by a nonincreasing sequence of discrete stopping times.

1.3 Proposition and suppose that and define

For n = 1, 2, . . . , let 0 = r: < t l < * - * and limk-rn tl: = 00, sup&+ - I;) = 0. Let I be an {F,+}-stopping time

52 STOCHASTIC PRCK€SSES AND MAWlNCALEs

Then t, is an {S,}-stopping time and limndm 7, = 7. If in addition {I:} t {t;"), then t, 2 tn+l.

Recall the intuitive description of 9, as the information known to an observer at time t. For an (9,)-stopping time 7, the a-algebra 9, should have the same intuitive meaning. For technical reasons S, is defined by

(1.3)

Similarly, PC+ is defined by replacing 9, by 9,+. See Problem 6 for some motivation as to why the definition is reasonable. Given an E-valued process X, define X(a0) c xo for some fixed xo E E.

9, = { A E 9: A n ( 7 s t } E 9, for all t 2 0).

1.4 Proposition Let t and u be {9,}-stopping times, let y be a nonnegative 9,-measurable random variable, and let X be an ($r,}-progressive E-valued process. Define X' and Y by Xr(r) = X(7At ) and Y(t) = X(7 + r), and define 9, = F I h , and MI = f,,,, t 2 0. (Recall that r h t and .c + r are stopping times.) Then the following hold:

(4 .Fr is a u-algebra. (b) T and 7 A u are SP,-measurable. (c) If t 5 us then F, c F.. (d) X ( t ) is fr-measurablc. (e) {Y,} is a filtration and X' is both {gJ-progressive and

(f) {Ju;) is a filtration and Y is {J1PIj-progressive. 0 7 +- y is an {fJ-stopping time.

{#,}-progressive.

Proof. (3 Clearly 0 and 0 are in PI, since 9, is a u-algebra and {r 5 I } E F,. If A A (7 S c } EP,, then A' n {t s, t } = (t 5 t ) - A n (7 s t ) E .F,, and hence A E implies A' B 9,. Similarly Ak A {s 5; t } E s,, k = I , & . . . , implies ( U r A , ) n (7 s t } = U&(Ak n {T I; t } ) E S,, and hence f, is closed under countable unions.

(1.4) { T A U s c } n {T s r } = { T A U 5 c A t } n {T s r }

(b) For each c 2 0 and t 2 0,

= ( { T 5 c A t } u {a I; c A r j ) n (t 5 t ) E F,.

Hence { f A u 5 c] E .Fr and r A d is S,-measurable, as is 7 (take u = 1).

1. STOCHASTIC moassEs 53

(c) If A E .Ft, then A n {a S t } = A n { t < t } n {IT s t } E 9, for all r 2 0. Hence A E 9#.

(d) Fix t 2 0. By (b), T A t is .F,-measurable. Consequently the mapping o - r ( t ( o ) A r , o) is a measurable mapping of (a, 9,) into ([O, r ] x Q, a[O, t ] x 9,) and since X is IF,}-progressive, (s, a)-+ X(s, w) is a measurable mapping of ([0, t ] x R, a[O, t3 x 9,) into ( E , 1(E)). Since X ( t A t ) is the composition of these two mappings, it is .F,-measurable. Finally, for P E @E) , { X ( r ) E r} n { 7 s t } = { X ( T A I ) E T} n { T s t } E .F, and hence

By (a) and (c), (Y,} is a filtration, and since 9, c 9, by (c), X' is (9,)-progressive if i t is {Y,}-progressive. To see that X' is (Y,}-progressive, we begin by showing that if s 5 t and H E a[O, t ] x .Fs, then

(1.5) H n (10, t ] x { T A t 2 S } ) E taco, t ] x Flh, = a[O, 13 x 9,.

To verify this, note that the collection X',,, of H E: a[O, t ] x 9, satisfying (1.5) is a a-algebra. Since A E 9, implies A n { T A t 2 s} E F,,,, it follows that if B E a[O, 13 and A E 9,. then

(1.6) (B x A) n ([0, t ] x { T A C 2 s))

{x(t) E rj E 9,. (el

= B x ( A n { T A C z s}) E a[O, C] x Y,,

so B x A E a[O, r ] x YS.

(1.7) {(s, W ) E LO, t3 x R: x ( T ( ~ ) A ~ , 0)) E r}

But the collection of B x A of this form generates

Finally, for r E d ( E ) and t 2 0,

= {(S,W):X(T(W)AS,W)E~, ~ ( w ) A r 5;sst)

= ({(s, w ) : T (w)A t 5 s 5; I } n ( [ O , r ] x { X ( T A I ) E r})) u {(s, 0): x(s, E r-, s < t(w) A C)

since

(1.8) {(s, 0)): ~ ( w ) A t I, s t }

and since the last set on the right in (1.7) i s in a[O, I ] x Y, by (1.5). (0 Again {HI} is a filtration by (a) and (c). Fix r 2 0. By part (e) the

mapping (s, u)-+ X((t(w) + t ) A s , w) from ([O, 003 x Q, a [ O , 003 x F,,,)

54 STOCHASTIC CIIOCESSLS AND MARTINGALES

into (E, @E)) is measurable, as is the mapping (u, a)-+ (r(w) + u, 0) from ([O, t] x fi, a[O, t] x gFt+J into ([0, 003 x Q, S[O, 003 x gr+J. The mapping (u, a)+ X(T(O) + u, o) from ([O, t ] x Q, a[O, r ] x Yr+J into (E, A?(&) is a composition of the first two mappings so it too is measurable. Since Z1 = F,+, , Y is {X1j-progressive,

C@ Let y. = [ny]/n. Note that ( 7 + y. s t } n { y , = k/n} = { T 5 t - k/n} n (7. = k/n} E 91-r,m, since (7. = k /n} E 9,. Consequently, { T + y. S t } E 9,. Since 7 + y = SUPAT + 7,). part (g) follows by Proposi- tion 1.2(b). 0

Let X be an E-valued process and let r E S(E) . Thefirst entrunce time into r is defined by

(1.9) Te(I‘) = inf ( t : X( t ) E r} (where inf 0 = m), and for a [O, m]-valued random variable a, the first entrance time into I‘ after u is defined by (1.10) Te(r, 0) = inf {t 2 u: X(r) e r}. For each w r s n and O S S 5 t, let Fx(s, t , w ) c E be the closure of { X(u, a): s ,< u I; t}. Thejrst contact time with r is defined by

(1.1 1) Tc(r) = inf { t : F,(O, t) n r # 0) and thejrst contact time with I’ after a by

(1.12) q(r, a) = inf { t 2 a: Fx(a, I ) n r it a}. The first exit time from r (after a) is the first entrance time of Iy (after cr). Although intuitively the above times are “recognizable” to our observer, they are not in general stopping times (or even random variables). We do, however, have the following result, which is sufficient for our purposes.

1.5 Proposition Suppose that X is a right continuous, {.F,}-adapted, E- valued process and that d is an {@,}-stopping time.

(a) If r is closed and X has left limits at each t > 0 or if r is compact,

(b) If r is open, then re(r, 0) is an (b,+)-stopping time. then Tc(r, a) is an {4tl}-stopping time.

Proof. Using the right continuity of X, if r is open, (1.13) {t,(r, U ) < t ) = u {x(s) E r) n {U c S} E F,,

a 6 0 n l O . 0

implying part (b). For n = 1. 2,. . . let r, = {x: r(x, r) < l/n}. Then, under the conditions of part (a), zc(I‘, Q) = limn-m re(r,, , a), and

(1.14) {rc(r.4s-r} ~ P ( ( ~ ~ t } n { x ( t ) ~ r } ) u n . { ~ ~ ( r . , a ) < t } ~ ~ ~ . o

2 mnwAus 55

Under slightly more restrictive hypotheses on {F,}, a much more general result than Proposition 1.5 holds. We do not need this generality, so we simply state the result without proof.

1.6 Theorem Let IS,} be complete and right continuous, and let X be an E-valued {St,)-progressive process. Then for each r E @E), r , ( Q is an ( 9,)-stopping time.

Proof. See, for example, Elliott (1982). page 50. 0

2. MRTINGALES

A real-valued process X with E[IX(t)lJ e 00 for all r 2 0 and adapted to a filtration (S,} is an {.%,}-martingale if

(2.1) ECX(t + s) I S,] = X(r), t , s 2 0.

is an {SF,}-submarringale if

(2.2) ECWt + s)l.!FJ 2 XO), r , s 2 0,

and is an {SFIP,)-supermartingak if the inequality in (2.2) is reversed. Note that X is a supermartingale if - X is a submartingale, and that X is a martingale if both X and - X are submartingales. Consequently, results proved for submartingales immediately give analogous results for martingales and supermartingales. If {@,} = {Sf} we simply say X is a martingale (submartingale, supermartingale).

Jensen's inequality gives the following.

2.1 Proposition (a) Suppose X is an {.!F,}-martingale, cp is convex, and &[lcp(X(t))(] < 00 for all t 2 0, Then cp 0 X is an (9,}-submartingale.

(b) Suppose X is an (9,)-submartingale, cp is convex and nondecreasing, and &[lcp(X(t))l] < 00 for all t 2 0. Then cp 0 X is an (9,)-submartingale.

Note that for part (a) the last inequality is in fact equality, and in part (b) the 0 last inequality follows from the assumption that cp is nondecreasing.

2.2 Lemma Let r , and r2 be {.!F,}-stopping times assuming values in Itl, t 2 , . . . , tm} c [O, 00). If X is an {9,}-submartingale, then

(2.4) ECX(r2)lFI,1 2 x(fl A rd.

56 STOCHASTIC PROCESSES AND MARTINGALES

Proof. Assume t, K tz < . * * < t,. We must show that for every A E 9,,

(2.5)

Since A = uys l(A n ( t l = ti}), it is sufficient to show that n

The following is a simple application of Lemma 2.2. Let x t = x VO,

2.3 Lemma Let X be a submartingale, T > 0. and F c [O, TJ be finite. Then for each x > 0,

(2.9) I; x - ' E [ X + ( T ) ]

and

Proof. Then

Let T = min { r E F: X ( t ) 2 x) and set T , = T A T and T , = T in (2.4).

(2.1 1) E[X(T)] 2 E [ X ( t A 7'11 = E C X ( T ) Z l r < m J + ECX(T)Xir=a)I*

and hence

which implies (2.9). The proof of (2.10) is similar. 0

2. MARTINGALES 57

2.4 Corollary Let X be a submartingale and let F c [0, 00) be countable. Then for each x > 0 and 7' > 0,

(2.13) .( sup x( t )2 s x ~ E C X + ( T ) I t a F n ( 0 . T I

and

(2.14)

Proot Let F, c F, c . . . be finite and F = U F , . Then, for 0 < y < x,

(2.15)

P{ inf X ( r ) S - x s; .Y '(E[Xt(7)1 - E [ X ( O ) ] ) .

p{ sup x ( t ) 2 x s y - l ~ ~ ~ + ( r ) ~ .

r c F n (0 . TI

0 Letting y-' x we obtain (2.13), and (2.14) follows similarly.

Let X be a real-valued process, and let F c LO, 00) be finite. For a < h define r l = min { t E F : X ( t ) I a}, and for k = 1,2,. . . define ak = min { t > t k :

I E F, X ( r ) 2 h } and r k t , = min { t > a*: t E F, X(r ) < u } . Define (2.16) V(a, h, F) = max {k: ak < ao}.

The quantity V(a, b, F) is called the number of upcrossings of the interval (a, h) by X restricted to F.

I E F A 10. 7 1

2.5 lemma then

Let X be a submartingale. If T > 0 and F c [0, 7'1 is finite,

(2.17)

Proof. Since ut A T I rk + I A 7 , Lemma 2.2 implics

which gives (2.17).


2.6 Corollary Let X be a submartingale. Let T > 0. let F be a countable subset of [O, TI , and let F, c F , c - - be finite subsets with F = U F , , . Define V(a, b, F ) = lim,,-.m V(a, 6, FJ Then V(a, b, F) depends only on F (not the particular sequence (F, , } ) and

(2.19)

Proof. The existence of the limit defining V(a, 6, F) as well as the independence of V(a, b, F) from the choice of { F , , } follows from the fact that G c H implies V(a, 6, G) 5 U(a, b, H). Consequently (2.19) follows from (2.17) and the monotone convergence theorem. 0

One implication of the upcrossing inequality (2.19) is that submartingales have modifications with “nice” sample paths. To see this we need the following lemmas.

2.7 Lemma Let (E, r ) be a metric space and let x: [O,oo)--t €. Suppose x(t +) = lim,,,, x(s) exists for all r L 0 and ~ ( t - ) = lim,-l- x(s) exists for all t > 0. Then there exists a countable set r such that for f E (0,oo) -I-, x(t -) = x( t ) = x(t +).

Let r,, = { t : r(x(t-)l x(i))Vr(x(i-), x(t+))Vr(x(t), x(f+)) > n-I} . Then r,, n [0, T ] is finite for each T > 0.

Proof. Since we may take r = U,, r,, it is enough to verify the last statement. If r,, n [0, 7 3 had a limit point r then either x(t -) or x(r +) would fail to

0 exist. Consequently rm n LO, T ] must be finite.

2.8 Lemma Let (€, r ) be a metric space, let F be a dense subset of [0, a), and let x: F - r E. I f for each t 2 0

(2.20)

exists, then y is right continuous. If for each I > 0

(2.21) y - ( t ) = lim x(s) s - I - S S F

exists, then y - is left continuous (on (0, 00)). If for each r > 0 both (2.20) and (2.21) exist, then y-(f) = f i t - ) for all I r 0.

2. MARTINGALES 59

Proof. Suppose (2.20) exists for all f 2 0. Given to > 0 and E > 0, there exists a 6 > 0 such that r(y(to), x(s)) 5 E for all s E F n ( t o , to + a), and hence

(2.22)

for all s E ( t o , to + 6) and the right continuity of y follows. The proof of the other parts is similar. 0

Let F be a countable dense subset of [O, 00). For a submartingale X, Corollary 2.4 implies P { ~ u p ~ ~ ~ ~ ~ ~ , ~ ~ X ( t ) < cn) = 1 and P{infleFnlo,rlX(f) > - O D } = I for each 7'> 0, and Corollary 2.6 gives P( V(a, h, F n [ O , T I ) < a)) = I for all ci < h and T > 0. Let

(2.23) Ro = f i ({ sup X ( t ) < 00} n { inf X(r) > - m 1 n = l l c F n l O , n l I E F n lo. nl

n n (W, h, F n [O. t i ] ) < 0 0 ) a r h

a. b e 8

Then P(Qo) = 1 . For w E Ro ,

(2.24)

exists for all t 2 0, and

(2 .25 )

Y(r, w ) = lim X ( s , o) S - I t

.s E F

exists for all I > 0; furthermore, Y( ., o) is right continuous and has left limits with Y(i -, o) = Y - ( I , o) for all I > 0 (Problem 9). Define Y ( t , o) = 0 for all w 4 R, and t 2 0.

2.9 Proposition Let X be a submartingale, and let Y be defined by (2.24). Then r = ( 1 : P ( Y ( i ) # Y(i - )} > 0) is countable, P ( X ( r ) = Y ( r ) ) = 1 for r 4 r, and

(2.26)

defines a modification of X almost all of whose sample paths have right and left limits at all 2 0 and are right continuous at all t $ r.

Proof.

(2.27)

For real-valued random variables q and < (defined on (R, 9, P) ) define

y (q , t) = inf ( E > 0 : P ( ) q - tl > E } < E } .

Then y is a metric corresponding to convergence in probability (Problem 8). Since Y has right and left limits in this metric at all f 2 0, Lemma 2.7 implies r is countable.

Let a E R. Then X V a is a submartingale by Proposition 2.1 so for any T > 0,

(2.28)

and since ( E [ X ( T ) V a ( f f ] : 0 5 t 5 T) is uniformly integrable (Problem lo), i t follows that { X ( f ) V a : 0 I; t 5 T} is uniformly integrable. Therefore

(2.29) X ( r ) V a I lim E [ X ( s ) V a 1 9 f ] i= E [ Y ( r ) V a ) 9 ; ] , r 2 0.

a s X ( t ) v a s E [ X ( T ) V ~ ~ . % ; K ] , O s t s T ,

S + l +

S C Q

Furthermore if t f r, then

(2.30) E[E[Y(t)VaIF(P,Y] - X ( r ) V a ) I; lirn E [ Y ( r ) V a - X ( s ) V a ] = 0, s - I - s c Q

and hence, since Y ( t ) = Y(t -) as. and V(t -) is 9/-measurable,

(2.31) X(r) V a = EL Y( t ) V a J = Y(r) V a as.

Since a is arbitrary, P { X ( r ) = Y(r)) = 1 for t 6 r. To see that almost all sample paths of 8 have right and left limits at all

t 2 0 and are right continuous at all r 4 r, replace F in the construction of Y by F u r. Note that this replaces no by 0, c no, but that for o E no, Y( a , w ) and 8( a , w) do not change. Since for w E 0,

(2.32) Y(t . w ) = lim Y(s, w) = lim X(s, w), I 2 0, s-1+ s - I +

s a F u T

it follows that

(2.33)

which gives both the existence of right limits and the right continuity of 0

Y ( t , o) = Iim &s, w), r 2 0, s-I +

8( -, w) at t $ r. The existence of left limits follows similarly.

2.10 Corollary Let 2 be a random variable with EL I Z 11 < 00. Then for any filtration (9,) and t L 0, E[ZIsF,J-, E[Z19,+] in L' ass-+ I + .

Proof. Let X(r) = EIZJ.FIJ, I z 0. Then X is a martingale and by Proposi- tion 2.9 we may assume X has right limits as. at each r 2 0. Since ( X ( l ) } is uniformly integrable, X ( t + ) s lirn,,,, X ( s ) exists a.s. and in L! for all f 2 0.

2. MARTINGALES 61

We need only check that X(t + ) = E [ Z ( . 4 t l + ] . Clearly X ( r + 1 is .Fl + -measurable and for A E 9, + ,

(2.34)

hence X ( t +) = E [ Z I .Fl +I.

X ( t + ) d P = lim

0

2.11 Corollary I f {,Fl} is a right continuous filtration and X is an (9,)-martingale, then X has a right continuous modification.

2.12 Remark sample paths of a right continuous submartingale have left limits at all r > 0.

Proof. With reference to (2.24) and Corollary 2.10, for t < T,

(2.35)

I t follows from the construction of Y in (2.24) that almost all

0

Y ( r ) = lim X(s ) = lirn E [ X ( ’ I ’ ) I . f , ] s -*I t s - I + s c F r e F

= E / X ( T ) I . F , + J = E [ X ( T ) I . f , ] = X ( i ) as.,

so Y is the desired modification. 0

Essentially, Proposition 2.9 says that we may assume every submartingale has well-behaved sample paths, that is, if all that is prescribed about a submartingale is its finite-dimensional distributions, then we may as well assume that the sample paths have the properties given in the proposition. In fact, in virtually all cases of interest, I- = a, so we can assume right continuity at all t 2 0. We do just that in the remainder of this section. Extension of the results to the somewhat more general case is usually straightforward.

Our next result is the optional sampling theorem.

2.13 Theorem and f 2 be (f,}-stopping times. Then for each T > 0, (2.36)

If, in addition, r 2 is finite as., E[lX(r,)(] < 00, and

Let X be a right continuous {.f,}-submartingale, and let T ,

E[X(T, A T)14tr,3 2 X(r, A r2 A T ) .

(2.37)

then

2.14 Remark Note that if X is a martingale (X and - X are submartingales), then equality holds in (2.36) and (2.38). Note also that any right continuous {.f,}-submartingale is an { . f l + f-submartingale, and hence corresponding

U inequalities hold for (9, + )-stopping times.

62 STOCHASTIC PRocfsSES AND MAITINGALES

Proof. = (k f IM2" if k/2" rj

T~ < (k + 1)/2". Then by Proposition 1.3, $1 is an {$,}-stopping time, and by Lemma 2.2, for each a E: R and T > 0,

(2.39)

Since T,, t

(2.40)

Since Lemma 2.2 implies

(2.41) a 5 X ( r y ' A T ) V a S E [ X ( T ) V a l f , p I ,

{X(r!j'"A T ) V a j is uniformly integrable as is ( X ( ~ Y ' A T Y ' A T ) V a } (Problem 10). Letting n-+ 00, the right continuity of X and the uniform integrability of the sequences gives

For i = 1, 2, let T?) = a, if T~ = GO and let

E [ X ( t y ' T) V a I9',?,] 2 X(r',"' A T?' A T ) v a.

by Proposition I .4(c), (2.39) implies

E[X(ry' A 7') V a I T,,] 2 E[X(r:"'A 7$"A T ) V a I Sr,].

(2.42) E [ X ( r , A T ) V o l I S , , ] z E [ X ( r , At2AT)Va l .F r l l = X ( T I A T ~ A T ) V U .

Lerting a+ - 00 gives (2.36), and under the additional hypotheses, letting 0 T 3 00 gives (2.38).

The following is an application of the optional sampling theorem.

2.75 Proposition Let X be a right continuous nonnegative {~f}-superinartingale, and let r,(O) be the first contact time with 0. Then X ( r ) = 0 for all I 2 r,(O) with probability one.

proof. For 11 = 1.2 ,..., let T, = ~ ~ ( [ 0 , n - ~ ) ) , the first entrance time into C0,n-I). (By Proposition 1.5, 7, is an {9f+}-stopping time.) Then TJO) = limm+m T". I f T, < 00, then X ( T , ) 5 n - ' . Consequently, for every r 2 0,

(2.43) E C X ( l ) I $1. + 3 5 X ( t A TJr

and hence

(2.44) ECX(f)I~,"+lX,r,sf, 5 0 - l .

(2.45) ~ r ~ ( ~ ) X , I < , o , r r , l = 0.

Taking expectations and letting n-+ 00, we have

The proposition follows by the nonnegativity and right continuity. 0

Ncxt we extend Lemma 2.3.

1. MARTINGALES 63

2.16 Proposition (a) Let X be a right continuous submartingale. Then for each x > 0 and T > 0,

(2.46) X ( t ) 2 x I x- 'E[X+(T)] 1 and

P inf X ( r ) 5 - x I x ' '(E[X+(T)] - E [ X ( O ) ] ) . I S T

(2.47) { (b) Let X be a nonnegative right continuous submartingale. Then for

a > I and T > 0.

(2.48)

Proof. Corollary 2.4 implies (2.46) and (2.47), but we need to extend (2.46) in order to obtain (2.48). Under the assumptions of part (b) let x > 0, and define T = inf { I : X ( t ) > x). Then T i s an {4tl.)-stopping time by Proposition l.S(b), and the right continuity of X implies A'(?) 2 .Y i f T c 00. Consequently for T > 0,

(2.49)

and the three events have equal probability for all but countably many x > 0. By Theorem 2.13,

{;;! x(t) > x} c 5 T J c sup X(r ) 2 x , { i 6 J 1 (2.50)

and hence

E[X(r A T ) ] 5 E [ X ( T ) ] ,

(2.5 I ) x P { T 5 E[A'(T)X{ts T I ] 5 ECX(T)X(~.l'\ln

Let cp be absolutely continuous on bounded intervals of [0, m) with cp' 2 0 and cp(0) = 0. Define Z = supls X ( l ) . Then for p > 0,

(2.52) E[p(ZA/?) ] = [ p'(x)P(Z > x) d x

I cp'(x)x ~ 'ECX(77xlzr,,1 d.u

= E [ X ( T)&% A p)] where $(z) = f: v'(.u)x- ' dx.

64 STOCHASrlC PROCESSES AND MMTfNGMES

If cp(x) = xu for some a > 1, then

a a - 1 - E[X(T)"]"aE[(Z Af l~ ] 'u -" 'a ,

and hence

E[X(T)']''". a

E[ (Z A p)7"" 5 -- a - 1

(2.54)

Letting fi-, og gives (2.48). 0

2.17 Corollary Let X be a right continuous martingale. Then for x > 0 and T > 0,

(2.55)

and for a > 1 and T > 0,

P sup IX(t)l 2 x s X - ~ E [ I X ( T ) I ] , L s r (2.56)

Proof. Since 1x1 is a submartingale by Proposition 2.1, (2.5s) and (2.56) 0 follow directly from (2.46) and (2.48).

3. LOCAL MARTINGALES

A real-valued process X is an {9,}-local martingale if there exist {f,)-stopping times T' S r3 s * with zn+ 00 as. such that Xrn = X( * A t,,) is an {9,}-rnartingale. Local submartingales and local supermartingales are defined similarly. Of course a martingale is a local martingale. In studying stochastic integrals (Chapter 5 ) and random time changes (Chapter a), one is led naturally to local martingales that are not martingales.

3.1 Proposition If X is a right continuous {9,}-local martingale and t is an {f,}-stopping time, then X' = X( A t ) is an {f,}-local martingale.

Proof. There exist {S,f-stopping times s s - * . such that t,-+ 00 as. and X'" is an (.F',}-martingale. But then X'(. AT,,) = X'"(. A t ) is an

0 (S,}-martingale, and hence X' is an {4r,}-local martingale.

3. LOCAL minN<ims 65

In the next result the stochastic integral is just a Stieltjes integral and consequently needs no special definition. As before, when we say a process V is continuous and locally of bounded variation, we mean that for all w E fl, V( ., w ) is continuous and of bounded variation on bounded intervals.

3.2 Proposition Suppose X is a right continuous (.F,}-local martingale, and V is real-valued, continuous, locally of bounded variation, and {S,}-adapted. Then

(3.1) M(t) = [ V(s) dX(s) = V ( t ) X ( t ) - V(O)X(O) - X ( S ) dV(s) 1.' is an {.F,)-local martingale.

Proof. The last equality in (3.1) is just integration by parts. There exist (9,j-sropping times T , s r2 -< - such that T,,+ 00 as. and Xrn is an {Sc,}-martingale. Without loss of generality we may assume r,, s T,(( - 00, -n] u [n, a)), the first contact time of (- a, -n] u [n, m) by X . (If not, replace T,, by the minimum of the two stopping times.) Let R be the total variation process

where the supremum is over partitions of [O, t ] , 0 = so < s, < * . < s, = t . For n = 1,2,. . . let y,, = inf { t : R(t) 2 n}. Since R is continuous, y,, is the first contact time of [n, a)) and is an {SF,}-stopping time by Proposition 1.5. The continuity of R also implies y,, -+ 00 as.

Let 6, = ?,,AT,,. Then u,,+ 00 as. and we claim M( * At?,,) is an {.F,}-martingale. To verify this we must show

for all t , s 2 0. Let t = uo < uI < . . * < u, = t + s. Then

(3.4)

since Xu" is an (9,}-martingale and Vu"fu,) is Sm,-measurable. Letting max,lu,, I - uk1+ 0, the sum in (3.4) converges to the second integral in (3.3)


as. However, to obtain (3.3). we must show that the convergence is in f! Observe

(3.5) I I V""(uk)(X8I(uk+ ,) - Xea(uk)) k = O

-- vyr + s)X'qf + s) - Vuqo)xuqo)

m - 1

I - c X U " ( U k + 1 W V U ( I ( ~ k + , ) - VYU, ) ) (

k = O

5 I vyt + S)X'"(t + s) - V'I(0)XuyO)I m- I

k = O + c I X%k + I ) I I V Y h + I ) - V"(u*)I

s I vyt + S)X'qt + s) - v"~(0)XyO) I + (n v I XYl + s) I )R(a,).

The right side is in L', so the desired convergence follows by the dominated convergence theorem. 0

3.3 Corollary Let X and Y be real-valued, right continuous, {.Fl}-adapted processes. Suppose that for each I, infss, X(s) > 0. Then

(3.6) M,(r) = X(r) - 6 Y(s) ds

is an (9,}-local martingale if and only if

(3.7)

is an (9,)-local martingale.

Proof. Suppose M, is an (.Fl)-local martingale. Then by Proposition 3.2,

3. LOCAL MARTlNGALES 67

is an {9,}-local martingale. Conversely, if M, is an (9,}-local martingale, then

(3.9)

= X ( i ) - X ( 0 ) - Y(s) ds

is an {.f,}-local martingale. 0

We close this section with a result concerning the quadratic variation of the sample paths of a local martingale. By an "increasing" process, we mean a process whose sample paths are nondecreasing.

3.4 Proposition Let X be a right continuous {.F,}-local martingale. Then there exists a right continuous increasing process, denoted by [XI, such that for each f 2 0 and each sequence of partitions {u:"'} of [0, r] with max,(up! I - up') -b 0,

(3.10)

as n--. m. If, in addition, X is a martingale and & [ X ( C ) ~ ] < n3 for all r 2 0, then the convergence in (3.10) is in I!!.

Proof. Convergence in probability is metrizable (Problem 8); consequently we want to show that {xk(X(u:"!,) - X(up'))'} is a Cauchy sequence. If this were not the case, then there would exist E > 0 and (n , ) and {mi ) such that n,-r 00,

mi--+ m, and

(3.1 I )

for all i. Since any pair of partitions of [0, t3 has a common refinement, that is, there

exists ( u k } such that {up)} c ( u k } and {u:'"')) c {u,} , the following lemma con- tradicts (3.11) and hence proves that the left side of (3.10) converges in probability.

3.5 Lemma Let X be a right continuous {F,)-local martingale. Fix T > 0. For n = 1.2,. . . , let {@) and { o r ) } be partitions of [0, T ] with {or'} c {ur') and maxk(ur! - up') + 0. Then

68 STOCHASTJC HOCLsSES AND MARTINGALES

Proof. Without loss of generality we can assume X is a martingale (otherwise consider the stopped process P), and X(0) = 0. Fix M > 0, and let t = inf (s: (X(s)( 2 M or ( X ( s - ) l r: M}. Note that P { r s I } s E[ lX(r ) l ] /M by Corollary 2.17.

Let { u k ) and { u k } be partitions of [0, t ] , and suppose { u k } t {uk}. Let wk = max {v,: vl 5 uk}, and define qk s Xr(uk) - x'(uk- ,) and

(3.13) z = ~ ( x ' ( ~ , ) - X'(ul-1))' - c ( x ' ( u k ) - X'(uk-1))'

= c t k , where Ck = 2(xr(uk) - x'(uk- I))(X'(Uk- - Xr(wk - or ltkl s 4MlXr(uk) - X'(uk,l)( and that E[{k+l14t, ,J = 0. Consequently,

Note that either tk = 0

(3.14) m

is a discrete-parameter martingale. Let

(3.15) 1x1 s 4M2,

- 16M4, 1x1 > 4M'.

where the last inequality follows from the convexity of rp and the fact that for k 5 K, IX'(u,)l 5 M.

Using the fact that {Z,} is a discrete-parameter martingale,

3. LOCAL MARTINGAUES 69

Fix E > 0. Let a1 = min { u k : IX'(uk) - X'(ul)( 2 c } u { u l + ,} and /?, = u,+ Note that if vI = W k - l 5 U k - l u l t l r then (X'(u,. ,) - X'(w,. ,))' 5; cz. Consequently, by (3.16) and (3.17),

+ E C X ~ ~ ~ < ~ ~ + ,) 16M'(cp(Xr(Dl)) - v(X ' (a l ) ) ) l . I

Fix N 2 I , and let t = min {I: ~ ~ = O ~ f a i z , , i + , ) = N } . Let y = aL if L < 00 and y = T otherwise. Then y is an {$,}-stopping time, and hence by (3.18) and the convexity of cp,

Given E, E' > 0, let D = {s E [O, T I : I X ( s ) - X ( s - ) ) > 4 2 ) . Then there exists a positive random variable 8 such that s E D and s s t 5 s + 6 imply I X ( t ) - X(s)I 5 E', and 0 5 s < t s T , t - s s 8, and I W t ) - X(s)I 2 E imply (s, 13 n D # 0. Let ID1 denote the cardinality of D. On {max (u,+ I - ui) 5; S),

(3.20)

5 (101 A N ) 8 M 2 d .

Let S(T) be the collection of {F,)-stopping times a with a < T. Since

for all a E S(T), {cp(X'(a)): a E S(T)} is uniformly integrable. Consequently, the right side of (3.19) can be made arbitrarily small by taking E small, N large (so that PIN < IDI} is small), E' small, and max ( u , + , - 0,) small. Note that if N > [Dl and max ( u , + I - of) < S, then y = T .

70 STOCMNC PROC€SSES AND MMNNCALES

Thus, if Z(') is defined for {up ' } and {up)} as in (3.13), the estimate in (3.19) implies

(3.22)

which, since M is arbitrary, implies (3.12). 0

Proof of Proposition 3.4-conrinued. Assume X is a martingale and E[X(T)'] < 00. Let { U k ) be a partition of 10, TI, and let X' be as in the proof of Lemma 3.5. Then

(3.23) EIIc(x(uh+l) - x(uk))2 - c(xr(uk+,) - xr(uk))211

S EC(X(T) - X(TAr))21 + ECl(X(ug+ 1) - W)Mx(t) - X(UK))IXI~<TJ,

where K = max { k : uk < r) . Since for M 2 1, cp defined by (3.15) satisfies 1x1 s E + &)/E for every E > 0, the estimates in the proof of Lemma 3.5 imply { ~ ( X ' ( u : " : , ) - Xr(up)))l} is a Cauchy sequence in Ll (note that we need &[X(T) ' ] < 00 in order that this sequence be in L'). Consequently, since the right side of (3.23) can be made small by taking M large and max (uk + I - uk)

small, it follows that {c(X(iif ' i Convergence of the left side of (3.10) determines [ X ] ( t ) a.s. for each 1 2 0.

We must show that [ X ] has a right continuous modification. Since {Z,,,} given by (3.14) is a discrete-parameter martingale, Proposition 2.1 6 gives

- X(ut'))'} is a Cauchy sequence in L'.

(3.24)

Consequently, for 1 L n-, co and T > 0,

k S 2nT

and it follows that we can define [XI on the dyadic rationals so nondecreasing and satisfies

-3 0,

that it is

The right continuity of [XI on the dyadic rationals follows from the right continuity of X. For arbitrary t 2 0, define

(3.27)

Clearly this definition makes [XJ right continuous. We must verify that (3.10) is satisfied.

4. THE FROIECTION THEOREM 71

Let (uf} = {i/2": 0 5 i s [2"t]} u { t } . Then

-% 0,

and (3.10) follows. 0

3.6 Proposition Let X be a continuous (.Ff}-local martingale. Then [ X I can be taken to be continuous.

Proof. Let [ X I be as in Proposition 3.4. Almost sure continuity of [ X I restricted to the dyadic rationals follows from (3.26) and the continuity of X. Since [ X I is nondecreasing, it must therefore be almost surely continuous. (-J

4. THE PROJECTION THEOREM

Recall that an E-valued process X is (P,}-progressive if the restriction of X to [O, t ] x R is W[O, t ] x gf-measurable for each t 2 0, that is, if

(4.1 ) ((s, w): x(s, 0) E r-1 n "0, ti x 0) E a[o, CI x 9,

for each t 2 0 and r E ~ ( E ) . Alternatively, we define the a-algebra of { 9,) -progressiue sets W by

(4.2) W = { A E a[O, ao) x 9: A n ([0, t ] x Q) E a[O. t ] x 9,

for all t 2 0) .

(The proof that W is a a-algebra is the same as for 9, in Proposition 1.4.) Then (4.1) is just the requirement that X is a W-measurable function on [O, 00) x Q.

The a-algebra of {.F,}-oprional sets 0 is the a-algebra of subsets of [O,oo) x Q generated by the real-valued, right continuous {9,)-adapted processes. An E-valued process X is (9,}-optional if it is an @measurable function on LO, 00) x Q. Since every right continuous (9,)-adapted process is (*,}-progressive, 0 c W , and every (9rf}-optional process is { 9,) -progressive.

M STOCHASTIC fRocEuLE AND MARTINGALES

Throughout the remainder of this section we fix {SJ and simply say adapted, optional, progressive, and so on to mean {$,}-adapted, and so on. In addition we assume that (9,) is complete.

4.1 lemma Every martingale has an optional modification.

Proof. By Proposition 2.9, every martingale has a modification X whose sample paths have right and left limits at every c E [0, co) and are right continuous at every t except possibly for f in a countable, deterministic set r. We show that X is optional. Since we are assuming (6,) is complete, X is adapted. First define

(4.3) k + l

n n

(set X( - t/n) = X(0)). and note that adapted and right continuous, Y is optional.

(4.4)

x ( t ) E Y(r) = X(t -). Since Y. is

Fix E > 0. Define I,, = Oand, for n L= 0, 1,2,.. . , I , + ~ = inf {s > I,: IX(s) - X(s-)I > 6 or JX(s+) - X ( s - ) l > E

or J X ( s + ) - X(s)l > 6).

Since X ( s + ) = X(s) except for s E r,

(4.5) { I , c t } = u n u ii (Ix(r,) - WSJI > + 6 p, 1 n (~i . 11) I = 1

where {s,, t , ) ranges over all sets of the form 0 5; sl < f , < sz < t2 < * * * < s, < ti < t , I t , - s,I < l/m, and r , , sI E l- u Q. Define

m

a= I (4.6) U t ) = C xit., r. + l,rn)(t)~[~ XCr.) - x(r. - ) I > e r ( X ( t n ) - X ( T ~ - 1)

+ ~ ( l ~ ( I ~ - ~ ( I - } l > ~ ~ ~ ~ ( f ) - W-)). Since X has right and left limits at each t E [O, a), I, = 00, and hence 2: is right continuous and has left limits. By (4.5). {I, < s} E 6, for s s t , and an examination of the right side of (4.6) shows that ZL(f) is 4t,-measurable. Therefore 2: is optional. Finally observe that I Y(r) + 1irnm+,, Z:(c) - X(t)l S

0 E, and since E is arbitrary, X is optional.

4.2 Theorem Let X be a nonnegative real-valued, measurable process. Then there exists a [O, a]-valued optional process Y such that

(4.7) E [ X ( t ) I 6 r l = Y(T)

4. THE FROjtcnON THEOREM 73

for all stopping times T with P{T < a} = 1. (Note that we allow both sides of (4.7) to be infinite.)

4.3 Remark Y is called the optional projection of X . This theorem implies a partial converse to the observation that an optional process is progressive. Every real-valued, progressive process has an optional modification. The optional process Y is unique in the sense that, if Yl and Y, are optional processes satisfying (4.7), then Y, and Y, are indistinguishable. (See Dellacherie and Meyer (1982). page 103.) 0

Proof. Let A e 9 and B E O[O, a), and let 2 be an optional process satisfying E [ x , , ) ~ , ] = Z(t). 2 exists, since E [ x , , ) S , ] is a martingale. The optional sampling theorem implies E[XA I gFr] = Z(t) . Consequently, xe(r)Z(t) is optional, and

(4.8) E C X I ( T ) X A i 9 t I = xe(r)Z(r).

Therefore the collection M of bounded nonnegative measurable processes X for which there exists an optional Y satisfying (4.7) contains processes of the form xS x,,, B E a[O, a)), A E 9. Since M is closed under nondecreasing limits, and X , , X z E M, XI 2 X 2 implies XI - X2 E M, the Dynkin class theorem implies M contains all indicators of sets in O[O, a)) x 9, and hence all bounded nonnegative measurable processes. The general case is proved by

0 approximating X by X A n , n = I , 2,. . . ,

4.4 Corollary Let X be a nonnegative real-valued, measurable process. Then there exists Y: [0, 00) x 10, oo) x Q4 [O, 00] , measurable with respect to W[O, 00) x 0, such that

(4.9) E[X(T + .$)I.(p1] = Y(s, t)

for all a.s. finite stopping times T and all s 2 0.

Proof. Replace X&) by Xa(t + s) in the proof above. 0

4.5 Corollary Let X: E x [O,m) x Q - b [0, m) be 9 ( E ) x 9[0, 00) x b- measurable. Then there exists Y: E x [0, oo) x Q-, [0, oo], measurable with respect to g(E) x 0, such that (4.10) ECX(x, ~ ) l s r l = Y(x, T)

for all 8.s. finite stopping times T and all x E E.

Proof. Replace Xs(t) by xdx, I), B E g ( E ) x W[O, a)), in the proof of Theorem 4.2. 0

The argument used in the proof of Theorem 4.2 also gives us a Fubini theorem for conditional expectations.

74 STOCHASTIC mocwEs AND t.wnNcALEs

4.6 Proposltion Let X: E x il-+ R be B(E) x *-measurable, and let p be a a-finite measure on a ( E ) . Suppose I E[IX(x)( ]p(dx) < 00. Then for every 6-

algebra 9 c I, there exists Y: E x 0- R such that Y is @ E ) x 9- measurable, Y(x) = E [ X ( x ) l 9 ] for all x E E, Y(x)(lr(dx) -c a3 a.s., and

(4.1 1)

4.7 Remark With this proposition in mind, we do not hesitate to write

(4.12) 0

Proof. First assume p is finite, verify the result for X = x P x A . B E a(€), A E 9, and then apply the Dynkin class theorem. The a-finite case follows by

0 writing p as a sum of finite measures.

5. THE DOOB-MEYER DECOMPOSITION

Let S denote the collection of all (Il}-stopping times. A right continuous (.Fl}-submartingale is of class DL if for each T > 0, {X(t.A T): t E S} is uniformly integrable. If X is an (9,)-martingale or if X is bounded below, then X is of class DL (Problem 10).

A process A is increasing if A( *, a) is nondecreasing for all UJ E z1. Every right continuous nondecreasing function a on [O, 00) with 40) = 0 determines a Bore1 measure p,, on [O, a) by p,,[O, t ] -- a(t). We define

when the integral on the right exists. Note that this is not a Stieltjes integral if f and a have common discontinuities.

5.1 Theorem Let {9,} be complete and right continuous, and let X be a right continuous (Sl}-submartingale of class DL. Then there exists a unique (up to indistinguishability) right continuous {.FI}-adapted increasing process A with A(0) = 0 and the following properties:

(a) M = X - A is an {S,}-martingale.

5. THE DOOI&MEVIER MCOkYOSITION 75

(b) For every nonnegative right continuous {F,)-martingale Y and every I 2 0 and t E S,

Y ( s - ) dA(s)] = E[sb^' Y(s) (5.2) E[lAr

= E [ Y(t A T ) A ( ~ A T)].

5.2 Remark (a) We allow the possibility that all three terms in (5.2) are infinite. If (5.2) holds for all bounded nonnegative right continuous {S,}-martingales Y, then it holds for all nonnegative (9,)-martingales, since on every bounded interval [O, T ] a nonnegative martingale Y is the limit of an increasing sequence { V , ) of bounded nonnegative martingales (e.g., take U, to be a right continuous modification of Y:(t) = E [ Y ( T ) A

(b) If A is continuous, then the first equality in (5.2) is immediate. The second equality always holds, since (assuming Y is bounded) by the right continuity of Y

(5.3)

n I ~ , 3 .

+ n - ' 1 ( A (k ; - ) - A ( ? - ) ) ]

= E [ y ( t A ~ ) A ( l h ? ) ] .

The third equality in (5.3) follows from the fact that Y is a martingale. Property (b) is usually replaced by the requirement that A be pre-

dictable, but we do not need this concept elsewhere and hence do not introduce it here. See Dellacherie and Meyer (1982), page 194, or Elliot (1 982), Theorem 8.1 5. 0

(c)

Proof. For each E > 0, let X 8 be the optional projection of E - ' yo X( * + s)ds,

(5.4)

Then X, is a submartingale and

(5.5) lim E[ IX, ( t ) - X(t)(J = 0, t 2 0. 8 - 0

76 STOCHASTIC PRocEssEs AND MARTINGALES

Let V, be the optional projection of E- ' (X( * + E ) - X( )), and define

Since X is a submartingale,

(5.7) YJr) = E - ' E [ X ( t + E ) - X(t) I S,] 2 0,

and hence A, is an increasing process. Furthermore

(5.8) M, = X , - A,

is a martingale, since for t , u 2 0 and 5 E .IF, ,

(5.9) (MAC + 4 - M,(t)) dP

- j, E- ' (X(S + e) - X(s)) ds dP = 0. ) We next observe that {A,( t ) : 0 < e S 1) is uniformly integrable for each

t 2 0. To see this, let T: = inf (s: A,(s) 2 A}. Then

(5.10) E[Aa(t) - 1 A A&)] = E[A,(t) - A,(tfr A t ) ]

= E[X,( t ) - x,(t: A t)]

= EC&; < ,) (X, ( t ) - XXTf A t))I

Since P(T: c t ) 5; A-'E[A,(t)] 5 I - ' E [ X ( t + e) - X(O)], the uniform intcgra- bility of ( X ( r A ( t + 1)): t E S} implies the right side of (5.10) goes to zero as A+ 00 uniformly in 0 < E s 1. Consequently {A, ( t ) : 0 < E s 1) is uniformly integrable (Appendix 2). For each t L 0, this uniform integrability implies the existence of a sequence {em} with E. -+ 0, and a random variable A(t) on (a, 9) such that

(5.1 1)

for every E E f (Appendix 2). By a diagonalization argument we may assume the same sequence {e,) works for all t E Q n [O, 00).

Let 0 s s < t , s, c E Q, and B = {A( t ) < A(s)). Then

(5.12) E[(A(t) - A(S))X,I = lim EC(A,(t) - A,(~)lXBI 2 4 a-m

5. THE DOOU-MEYER OECOMMSlTlON 77

so A(s) s A(t) as. For s, t L 0, s, t E Q, and B E F,,

(5.13) E [ ( X ( f + s) - A(f + s) - x(f) + A(t ) )XB]

= lim E[(M,"(t + s) - M,m(t)),ysJ = 0, n - m

and defining M(c) = X ( t ) - A(t) for t E Q n [O, m), we have € [ M ( t + s ) ( 9 , ] = M ( t ) for all s, t E Q n LO, 00). By the right continuity of (9,; and Corollary 2.1 1, M extends to a right continuous (.F,)-martingale, and it follows that A has a right continuous increasing modification.

To see that (5.2) holds, let Y be a bounded right continuous {SF,)-martingale. Then for t L 0,

(5.14) E[Y( t )A( t ) ] -- lim E[ Y(t)A,l(t)J a-+m

= lirn E [ l Y ( s - ) dAJs)] n-m

and the same argument works with t replaced by t A r. Finally, to obtain the uniqueness, suppose A l and A, are processes with the

desired properties. Then A , - A, is a martingale, and by Problem 15, if Y is a bounded, right continuous martingale,

(5.15)

= Y(t)A,(Ol.

Let B = { A , ( t ) > A2(t ) } and Y(s) = E [ X ~ J S , ] (by Corollary 2.11, Y can be taken to be right continuous). Then (5.15) implies

(5.16) EC(A l(t) - AZ(t ) )XB] = 0.

Similarly take B = { A # ) > A&)} and i t follows that A # ) = A,(t) as. for each t L 0. The fact that A , and A, are indistinguishable follows from the right continuity. 0

78 STOCHASTlC ?ROCEESES AND MARTINGALES

5.3 Corollary If, in addition to the assumptions of Theorem 5.1, X is continuous, then A can be taken to be continuous.

Proof. Let A be as in Theorem 5.1. Let a > 0 and 7 = inf { t : X(t ) - A(r) [ -a , a ] } , and define Y = A( - A r) - X( A T ) + a. Since X is continuous, Y 2 0, and hence by (5.2),

(5.17) .[LA' Y(s - ) dA(s)] = Y(s) dA(r)], t 2 0.

For 0 5 s 5 I, Y(s- ) s a, and hence (5.17) is finite, and

(5.18) .[IAr (Y(s) - Y ( s - ) ) dA(s)]

0

= .[IAt (A@) - A@-)) dA(s)] = 0, t 2 0.

Since a is arbitrary, it follows that A is almost surely continuous. 0

5.4 Corollary Let X be a right continuous, {.W,}-local submartingale. Then there exists a right continuous, {9,}-adapted, increasing process A satisfying Property (b) of Theorem 5.1 such that At s X - A is an {.F,}-local martingale.

Proof. Let tl s T~ S * * * be stopping times such that 7,,+ oo and Xrn is a submartingale, and let yn = inf { t : X(t) s -a}. Then XraAvn is a submartingale of class DL, since for any {Ft}-stopping time T,

(5.19) X"" '"(7') A ( - n ) S X" A '"(TA T) < E[XrnAY"(T)I Sr].

Let A, be the increasing process for XraAya given by Theorem 5.1. Then A = limn+m A,, . 0

6. SQUARE INTEGRABLE MARTINGALES

Fix a filtration (9,). and assume {S,} is complete and right continuous. In this section all martingales, local martingales, and so on are (f,}-martingales, {PJ-Iwal martingales, and so on.

A martingale M is square integrable if E[1 M(t)I2] < 00 for all t 2 0. A right continuous process M is a local square integrable martingale if there exist stopping times T, 4 f l s - * such that 7,,+ 00 as. and for each n 2 I, Mh i M ( . h r , , ) is a square integrable martingale. Let A denote the collection of right continuous, square integrable martingales, and let .Aloc denote the collection of right continuous local square integrable martingales. We also need to

6. SQUARE INTEGRABLE MARTINGALES 79

define At, the collection of continuous square integrable martingales, and At, lot, the collection of continuous local martingales. (Note that a continuous local martingale is necessarily a local square integrable martingale.)

Each of these collections is a linear space. Let r be a stopping time. If M E A(Alo, , A,, A,, then clearly M' = M( A r) E A (Aloc, A,, A,. IOC ).

6.1 Proposition martingale).

If M E A(AlOc), then M 2 - [ M I is a martingale (local

Proof. Let M E A. Since for c, s 2 0 and t = uo c u, c . . < u, = t + s,

(6.1) E [ M ' ( t + S) - M'(t) I .F1] = E[(M(t + s) - M(t))' I PI]

the result follows by Proposition 3.4. The extension to local martingales is immediate. 0

If M E A(Aloc), then M 2 satisfies the conditions of Theorem 5.1 (Corollary 5.4). Let (M) be the increasing process given by the theorem (corollary) with X = M'. Then M 2 - ( M ) is a martingale (local martingale). If M E A~(.Mc,loc), then by Proposition 3.6, [ M I is continuous, and Proposition 6.1 implies [ M I has the properties required for A in Theorem 5.1 (Corollary 5.4). Consequently, by uniqueness, [ M I = ( M ) (up to indistinguishability).

For M, N E Aloc we define

(6.2) [ M , N ] = f ( [ M + N , M + N ] - [ M , M ] - [ N , N ] )

and

(6.3) ( M , N ) = f ( ( M + N , M + N ) - ( M , M ) - ( N , N ) ) .

Of course, [ M , N ) is the cross uuriation of M and N , that is (cf. (3.10)),

(6.4) [ M , N # t ) = lim 1 (M(uPi ,) - M(uP)))(N(u:"l,) - N(uf')) s-rm k

in probability. Note that [ M , M ] = [ M I and (M, M ) = ( M ) . The following proposition indicates the interest in these quantities.

6.2 Proposition If M , N E A (.,NloC), then M N - [ M , N ) and M N - ( M , N ) are martingales (local martingales).

80 STOCHASTIC PROCESSES AND MUTINGALES

Proof. Observe that

(6.5) M N - [M, N] =&(M + N)’ - [M + N, M + IV] - (M’ - CMI) - (N2 - CNI)),

and similarly for MN - (M, N). 0

If (M, N) = 0, then M and N are said to be orthogonal. Note that (M, N) = 0 implies MN and [M, N] are martingales (local martingales).

7. SEMICROUPS OF CONDITIONED SHIFTS

Let {sf} be a complete filtration. Again all martingales, stopping times, and so on are {.F,)-martingales, {f,}-stopping rimes, and so on. Let Y be the space of progressive (i.e., {9,}-progressive) processes Y such that sup, ELI Y(r)I] c a. Defining

(7.1)

and JV = { Y E 9: 11 Y 11 = 01, then 9 /Jv (the quotient space) is a Banach space with norm 11 * 11 satisfying the conditions of Chapter I , Section 5, that is, (7.1) is of the form (5.1) of Chapter 1 (r = (6, x P : r E [O, a)}). Since there is little chance of confusion, we do not distinguish between 14 and Y / N .

We define a semigroup of operators (.T(s)} on 9 by

(7.2) .F(s)Y(t) = E[Y(t + s)lS,J

By Corollary 4.4, we can assume (s, t, a)+ f ( s ) Y ( t , o) is a[O, a) x 8- measurable. The semigroup property follows by

(7.3) w)aS)Y(t) = U E C Y(t + + 4 I f , + “1 I $,I = E[Y(t + u + S ) I 9 , ]

= Y ( u + S)Y(t). Since

(7.4) SUP ECIf‘(s)Y(r)ll s sup ECI Y(t)lI, f I

{Y(s)} is a measurable contraction semigroup on 9.

surablefwith jt ls(~) I du < 00 and Z E 9’ by (5.4) of Chapter l, Integrals of the form W = Ef(u)Y(u)Z du are well defined for Bore1 mea-

(7.5)

7. SEMICROUPS OF CONDITIONED S H l R s 81

and

(7.6)

Define

Since ( Y , 2) E .G? if and only if

.T(s )Y = Y + Y(u)Z du, s 2 0, (7.8)

s? is the full generator for ( f ( s ) } as defined in Chapter I , Section 5. Note that the "harmonic functions", that is, the solutions of J Y = 0, are the martingales in 9.

7.1 Theorem The operator 2 defined in (7.7) is a dissipative linear operator with 910. - d ) = Y for all A > 0 and resolvent

(7.9) ( A - d ) - ' w = 5. e-".Y(s)W ds.

The largest closed subspace Yo of 9' on which (Y(s)} is strongly continuous is the closure of B(d), and for each Y E Yo and s 2 0,

L

m

(7.10)

Proof. (Cf. the proof of Proposition 5.1 of Chapter I .) Suppose ( Y, Z) E 2. Then

(7.11) e-"Y(sHLY - Z)(r) ds

= lm e-"E[AY(r + s ) - Z(r + s)J@,] ds


The last equality follows by interchanging the order of integration in the second term. This identity implies (7.9), which since $(s) is a contraction, implies d is dissipative. To see that @(A - d ) = 9, let W E 9, Y 5: e-"S(s)W ds, and

2 = LY - W. An interchange in the order of integration gives the identity

(7.12) Z ( r ) Y = [ Ae-AJ S ( r + u)W du ds

= I" Ae-AJ [ + ' f ( u ) W du ds,

and we have

(7.13) l . F ( u ) Z du = Ae-As .T(s + u)W du ds - S ( u ) W du l = [ Ae-As [+' .T(u)W du ds - S ( u ) W du.

Subtracting (7.13) from (7.12) gives

(7.14) Y ( r ) Y - f ( u ) Z du = la Ae-A* 1 J ( u ) W du ds

- l 2 e - L y ( u ) ~ du t/s + l . ~ t u ) W du

= Irn e-Au.T(u)W du - ( I - e-")S(u)W du

+ T ( u ) W du = Y,

which verifies (7.8) and implies (Y, 2) E d. e-"F(s)Wds E B(d) and lirn,..,Il," e-"F(s)W ds

= W (the limit being the strong limit in the Banach space 9). If (Y, 2) E d, then

If W E 9,,, then 1

and hence 9(.2) c Yo. Therefore 4po is the closure of 9(&. Corollary 6.8 of Chapter 1 gives (7.10). 0

7. SEMlCROUPS OF CONMTIONEO SHIFTS 83

The following lemma may be useful in showing that a process is in S(d).

7.2 lemma Let Y, Z , , Z2 E Y and suppose that Y is right continuous and that Z , ( t ) s Z,(r) as. for all t . If Y(r) - fo Z , ( s ) d s is a submartingale, and Y ( t ) - yo Z,(s) ds is a supermartingale, then there exists 2 E 9 satisfying Z , ( t ) s Z ( t ) I, Z,(t) a.s. for all c L 0, such that Y ( t ) - fo Z(s) ds is a martingale.

7.3 Remark The assumption that Y is right continuous is for convenience only. There always exists a modification of Y that has right limits (since Y ( t ) - Po Z,(s) ds is a supermartingale). The lemma as stated then implies the existence of Z (adapted to {9c,,}) for which Y(r +) - fo Z(s)ds is an {.%,,)-martingale. Since E[Y(t+)(4F,] = YO) and E [ j : + ' Z ( s ) d s I S , ] = E [ j : + ' E [ Z ( s ) J F J J dsISe,], Y(r) - fo E[Z(s)ISe,Jds is an (Sc,}-martingale. 0

Proof. Without loss of generality we may assume Z , = 0. Then Y is a submartingale, and since Y V 0 and (ro Z,(s) ds - Y(r))V 0 are submartingales of class DL, Y and ro Z,(s)ds - Y( t ) are also (note that I Y(r)J s Y(r)VO + (yo Z2(s) ds - Y(r)) VO). Consequently, by Theorem 5.1, there exist right continuous increasing processes A I and A, with Property (b) of Theorem 5.1 such that Y - A, and Y( t ) - Po Z,(s)ds + A,(t) are martingales. Since Y + A, is a submartingale of class DL, and Y + A, - (A , + A,) and Y( t ) + A,(f) - Po Z,(s) ds are martingales, the uniqueness in Theorem 5. I implies that with probability one,

(7.16) A&) + A , ( f ) = Z,(s) ds, t 2 0.

Since A, is increasing,

A ,(f + u) - A s Z,(s) ds, t , u 2 0,

0

ltM (7.17)

so A , is absolutely continuous with derivative Z, where 0 s Z s Z,.

7.4 Corollary If Y E 9, Y is right continuous, and there exists a constant M such that

(7.18) I E [ Y ( t + s) - Y ( t ) IS,] I 5 Ms, t. s 2 0,

then there exists Z E 9' with lZl s M as. such that Y(r) - So Z(s )ds is a martingale.

Proof. Take Z , ( d = - M and Z2(t) = M in Lemma 7.2. 0

7.5 Proposition Let Y E 9 and let t be the optional projection of Y( + s)ds and q the optional projection of Y(. + 6) - Y, that is, t(r) =

E [ l t Y( t + s) ds I S,] and q(t) = E[ Y(t + 6) - Y ( t ) I S,]. Then (C, q) E d.

84 STOCHASTIC FROCESSES AND MARTINGALES

Proof. This is just Proposition 5.2 of Chapter 1. 0

7.6 Proposition Let < E 49. If {s- 'E[t((r + s) - {(t)l4F,]: s > 0, f 2 0) is uniformly integrable and

(7.19) ~ - ' E [ < ( c + s) - WIS-,]A&) as s-o+, a.e. t ,

then (C, q) E 2.

Proof. Let <,(r) = E - 'E[' t(t + s) ds I S,] and q,(t) = E - ' E [ t ( t + E )

- ( ( t ) I F,]. Then (t,, q,) E d and as E--, 0, C,(t)-+ t(f) and

(7.20)

in L! for each.t L 0. 0

We close this section with some observations about the relationship between the semigroup of conditioned shifts and the semigroup associated with a Markov process.

For an adapted process X with values in a metric space ( E , r), let YU be the subspace of Y of processes of the form { f (X(t ) , t ) } , where S E E(E x [0, OD)),

and let YUo be the subspace of processes of the form { j ( X ( t ) ) } , f E B(E). Then X is a Markov process if and only if S(s): .&+ YU for all s 2 0, and it is natural to call X temporally homogeneous if Y(s): A,--, do for all s 2 0.

Suppose X is a Markov process corresponding to a transition function f ( s , x, r), define the semigroup { T(t)] on B(E) by

(7.2 1)

and let b denote its full generator. Then for Y Z ~ O X E A0,

(7.22) S(s )Y = T(s ) j 0 x, s 2 0,

and for U; h) E A,(Jo X , h 0 X ) E 2.

8. MARTINGALES INDEXED BY DIRECTED SITS

In Chapter 6, we need a generalization of the optional sampling theorem to martingales indexed by directed sets. We give this generalization here because of its close relationship to the other material in this chapter.

8. MARTINGALES INDEXED BY DIRECTED SETS 85

A set I is partially ordered if some pairs (u, u) E f x 9 are ordered by a relation denoted u < u (or u 2 u ) that has the following properties:

(8.1) For all u E I, u I u.

(8.2) If u s, u and v 5 u, then u = u.

(8.3) If u I u and us w, then II s w .

A partially ordered set J (together with a metric p on .P) is a metric lattice if (1, p) is a metric space, if for u, u E 9 there exist unique elements u A u E 9 and u V L: E 9 such that

(8.4) { w ~ f : w s u } n { w ~ f : wiu} = { W E # : w s u A u ]

and

(8.5) ( w E .f: w 2 u ) n { w E .f: w 2 u } = { w E 9: w 2 u v o } ,

and if (u, u)-+ u A u and (u, u)-+ LC V v are continuous mappings of .f x f onto 3. We write min { u , , . . . ,urn) for u , A * . . A u,, and max { u , , . . . , u,} for u , V . . V urn. We assume throughout this section that f is a metric lattice.

For u, u E f with u 5 u, the set [u, u] zs { w E f: u 5 w g u } is called an interval. Note that [u, u] is a closed subset of f. A subset F c f is separable from above if there exists a sequence {a,) t F such that w = limndm min {a i : w I a,, i s n} for all w E F. We call the sequence {a,,} a separating sequence. Note that F can be separable without being separable from above. Define f,, = { u E 9: u 5 u} .

Let (0, .F, P) be a probability space. As in the case .f = [O, a), a collection (9,) = {F,,, u E f} of sub-a-algebras of 9 is a jiltration if u 1; u implies 9c,, c 9,. and an 3-valued random variable r is a stopping time if ( r s u } E

9, for all u E .f. For a stopping time t,

(8.6)

{ A E 9: P(A) = 01 for all u E .f.

9, = { A E 9: A n { r S u ) E 9, for all u E 1).

A filtration {F,,} is complete if (a, 9, P) is complete and 9,, 2

Let rz = { v : infw,,p(u, w ) < n-'). We say that {P,,] is right continuous if

See Problem 20 for an alternative definition of right continuity.

8.1 Proposition hold :

Let r , , r2,. . . be (F,}-stopping times. Then the following

(a) maxks,,rk is an IS,}-stopping time. (b) Suppose {9,} is right continuous and complete. If t is an .#-valued

random variable and T = r , a.s., then r is an {SE,)-stopping time.


Proof. (a) As in the case .# = [O, a), {maxkj,rk s u ) = n k i r { ? k 5 u } E SY. (b) By the right continuity,

and hence

(8.9)

8.2 Proposition Suppose r is an {.F,,}-stopping time and a E f. Define

(8.10) on {T ~ a }

Then T* is an (F,)-stopping time.

8.3 Remark Note that ra is not in general equal to r A a, which need not be a stopping time. a

Proof. If u =a, 5 u ) = f'l E SF,. If Y s a, but u # a , then {r" S u ) = 0 { r s u} c 9,. In general, {r" 5 u } = {r" S u A a ) E 9,,, c 9,.

8.4 Proposition Suppose r is an {F,)-stopping time, a E f with T 5 a, and fa is separable from above. Let {a,} be a separating sequence for 3, with a, = a, and define

(8.1 1) r, = min {a,: r s a,, i s n} , n 2 1.

Then T , is a stopping time for each n 2 1, a = r , 2 T~ 2 ..., and T, = T .

Proof. Let F, be the finite collection of possible values of r,. For u e F, , (8.12) { r , = u ) = { T 5 u } n n { r s u } ~ E P,,

and in general

usF,nJ , u # u

(8.13) { r , s u ) = (J { r , = u } E 9,.

The rest follows from the definition of a separating sequence. V P F ,nJ,

0

Let X be an €-valued process indexed by 1. Then X is {91t.)-adapted if X(u) is 9,-measurable for each u E 9, and X is {F,}-progressive if for each u E 3, the restriction of X to f, x f'l is O(3,) x 9,-measurable. As in the

8. MARTINGALES INDEXED BY OllECTED S n S 87

case .f = [0, a), if J, is separable from above for each u E 3, and X is right continuous (i,e., lim,,,X(uVu, w) = X(u, w) for all u E f and w E Q) and (9,}-adapted, then X is {9,}-progressive.

8.5 Proposition Let t and d be {f,}-stopping times with r I n, and let X be { 9,)-progressive. Then the following hold :

(a) 9, is a a-algebra. (b) 9, c P,,.

(c) If .fu is separable from above for each u E f, then t and X(T) are .F,-measura ble.

Proof. The proofs for parts (a) and (b) are the same as for the corresponding results in Proposition 1.4. Fix a E 1. We first want to show that T* is 9,-measurable. Let (an} be a separating sequence for J, with a, = a, and define t; = min (a,: T' s a,, i g n). Then (8.12) implies r: is SiF,-measurable. Since limn-m t: = t", ro is 9,-measurable, and X(T") is 9,-measurable by the argument in the proof of Proposition 1.4(d). Finally, (t E P) n (T s a} = {T' E r) n {T s a ) E 9, for all a E f and r z a(f), so {T E P} E 9, and T is F,-measurable. The same argument implies that X ( r ) is F,-measurable. 0

8.6 Proposition Suppose {S,} is a right continuous filtration indexed by J. For each t 2 0, let T(C) be an {b,}-stopping time such that s s t implies r(s) I; t ( t ) and ~ ( t ) is a right continuous function of 1. Let H', = ft,l,, and let q be an (N,}-stopping time. Then r(q) is an {.%,}-stopping time.

Proof.

(8.14) {W S; u ) = u ({lr = I,} n {W 5 u ) ) .

Since { q = t , } E f,,,,, , { q = 1, ) n { t ( t I ) 5 u } E 9,. For general q approximate q by a decreasing sequence of discrete stopping times (cf. Proposition 1.3) and apply Proposition 8.I(b). 0

First assume q is discrete. Then

I

A real-valued process X indexed by J is an {f,}-martinga/e if E[IX(u)I] < m for all u E J, X is {*,}-adapted, and

(8.15)

lor all u. u E .f with u I u.

EC N o ) I P U I = X ( U )

8.7 Theorem Let f be a metric lattice and suppose each interval in f is separable from above. Let X be a right continuous (.F,}-martingale. and let r,

STOCHASTIC moasss AND MAITINCALES

and t2 be (SU}-stopping times with rI S T ~ . Suppose there exist {u,,}, {u,} c J such that

(8.16)

and

(8.17)

lim P{u, s tl 5 t2 s urn) = 1 n - m

Proof. Fix m 2 1 and for i = I, 2 define

(8.19)

Let {a,) be separating for [u,, urn] with a1 = u,, and define T:, = min (all: rIm < c f k , k n). Note T;, assumes finitely many values, and T ; , s rl,,,. Fix n, and let r be the set of values assumed by rl;, and T:,. For a E r with a # urn, {r;,,, = a) =I { r , 5 a} n {rl;,,, = a}, and hence A n {r;,,, = a) = A n {r l s a} n {r;,,, = a} E 9,. Consequently for a E r with a # urn,

(8.20) 1 X(r1,) d P A n It;. - 4

c ECX(UaJ) I 9#IX It;, I 8) d P -I ~ n [ t ; ~ - a l p a r

= z J X ( u m ) dP p c A n It;. - a) n It;, - #)

=I X(u,)dP A n It;, - a)

= I X(a) dP. A n It;,, - a;

Since slm = u, implies r;, = urn, (8.20) is immediate for a = urn, and summing over a E r, (8.20) implies

(8.21)

Letting n+ 00 and then m-+ a, gives

(8.22)

which implies (8.18). 0

9. PROBLEMS

I. Show that if X is right (left) continuous and {*,}-adapted, then X is { 9,} -progressive. Hint: Fix r > 0 and approximate X on [O, t ] x R by X, given by X&s, w ) = X ( t A ((Cnsl + l)/n). 4.

2. (a) Suppose X is E-valued and (9,)-progressive, and / E H E ) . Show that f o X is (9,}-progressive and Y ( t ) E yof(X(s)) ds is { 9,) -adapted.

(b) Suppose X is E-valued, measurable, and {PI}-adapted, and/€ B(E). Show that/. X is {9,}-adapted and that Y( t ) z fo/(X(s)) ds has an ($,)-adapted modification.

Let Y be a version of X. Suppose X is right continuous. Show that there is a modification of Y that is right continuous.

4. Let (E, r) be a complete, separable metric space. (a) Let <, , t., , . . . be E-valued random variables defined on (0, 9, P).

Let A = {a: lim t,,(w) exists}. Show that A E 9, and that for x E E,

3.

is a random variable (i.e., is f-measurable). (b) Let X be an E-valued process that is right continuous in probability,

that is, for each E > 0 and t 2 0,

(9.2)

Show that X has a modification that is progressive. Hint: Show that for each n there exists a countable collection of disjoint intervals [ t ; , s:) such that [O, ao) = u [ t : , s:) and

(9.3) P{r(X(lZ), X(s)) > 2-"} < 2-",

lim P{r(X(t) , X(s)) > E } = 0. $-+I +

f." s s < s:.

5. Suppose X is a modification of Y , and X and Y are right continuous. Show that X and Y are indistinguishable.

6. Let X be a stochastic process, and let T be a discrete {9:}-stopping time. Show that

(9.4)

Let (9,) be a filtration. Show that {9,+) is right continuous.

Let (Q, 9, P ) be a probability space. Let S be the collection of equivalence classes of real-valued random variables where two random vari-

S," = a(X(t A r ) : f 2 0).

7.

8.

90 STOCHASTIC PROCESSES AND MARTINCWS

ables are equivalent if they are almost surely equal. Let y be defined by (2.27). Show that y is a metric on S corresponding to convergence in probability.

9. (a) Let {x,} satisfy supnxn < OD and inf,x, > -00, and assume that for each a Oand w E Q,.

10. (a) Suppose X is a real-valued integrable random variable on (Q 9, P). Let r be the collection of sub-a-algebras of 9. Show that {ECX 191: 43 E r} is uniformly integrable.

(b) Let X be a right continuous {S,}-martingale and let S be the collection of {9,}-stopping times. Show that for each T > Oy { X ( T A T ) : T E S } is uniformly integrable.

(c) Let X be a right continuous, nonnegative {4F,}-submartingale. Show that for each T > Oy (X(TA7.r ) : T E Sj is uniformly integrable.

11. (a) Let X be a right continuous {~l}-submartingalc and T a finite {IP,}-stopping time. Suppose that for each c > 0, E[sup,,, [ X ( T + s) - X(r ) l ] < 00. Show that Y( t ) = X(T + t ) - X(T) is an (9,+,1-submattingale.

(b) Let X be a right continuous (9,)-submartingale and T , and be finite {.F,}-stopping times. Suppose 5 , 5 ' I ) and E[sup,(X((r, + s) A T ~ ) - X ( T , ) ( ] < 00. Show that E[X(r , ) - X(T, ) I .FJ 2 0.

12. Let X be a submartingale. Show that sup,E[lX(r)(] < OD if and only if SUP, E [ X ' ( t ) ] c= 00.

13. Let 4 and < be independent random variables with P { q = I } = P { q = - 1) = f. and E[JCJ] = a. Define

(9.5)

and

Show that X is an (..@,}-local martingale, but that X is not an {sC,X}-local martingale.

14. Let E be a separable Banach space with norm II * 11, and let X be an E-valued random variable defined on (a, 9, P).

9. mouws 91

(a) Show that for every E > 0 there exist {x,) c E and (6;) c 9 with B; n Bj = 0 for i #j,such that

(9.7) xe = C X r X e :

and show that

so that one can define

(9.10)

Extend Theorem 4.2 and Corollary 4.5 to bounded, measurable, E- valued processes.

E [ X 191 = lim E [ X , 191. 8 - 0

(c)

15. Let A , and A, be right continuous increasing processes with A,(O) = A,(O) = 0 and €[A#) ] < 00, i = 1, 2, t > 0. Suppose that A , - A2 is an {4C,}-martingale and that Y is bounded, right continuous, has left limits at each r > 0, and is {S,}-adapted. Show that

(9. I J ) Y(s - 1 d(A As) - A m

= Y ( s - ) dA,(s) - Y ( s - ) dA,(s)

is an {F,}-martingale. (The integrals are defined as in (5.1).) Hint: Let

(9.12) <(t) = 6 - 1 Y(s ) ds

and apply Proposition 3.2 to

(9.1 3)

16. Let Y be a unit Poisson process, and define M(r) = Y(r) - f . Show that M is a martingale and compute [MI and (M).

92 STOCHASTIC PRocEssEs AND MARTINGALES

17. Let W be standard Brownian motion. Use the law of large numbers to compute [WJ. (Recall [W] = < W) since W is continuous.)

18. Let 9' be the space of real-valued (9,)-progressive processes X satisfying 1; E[lX(t)12J dt < 00. Note that 2'' is a Hilbert space with inner product

(9.14)

and norm 11 X 11 = d m . Let A be a bounded linear operator on 2". Then A*, the adjoint of A, is the unique bounded linear operator satisfying ( A X , Y) = ( X , A'Y) for all X , Y E 3'. Fix s z 0 and let U(s) be the bounded linear operator defined by

( X , Y) = 1 ECXO)Y(t)l dr

(9.15)

What is U+(s)? (Remember that U*(s)X must be {PI}-progressive.) 19. Let M , , M ' , . . . , M , be independent martingales. Let 3 = [O, a)'" and

define m

(9.16) M(u) = n MXUi), u E #. 1 - I

(a) Show that M is a martingale indexed by Y. (b) Let 9, = u(M(u): u s u), and let ~ ( t ) , t 2 0, be {f,}-stopping times

satisfying r(s) 5 r(t) for s s t. Suppose that for each t there exists c, E 3 such that ~ ( t ) s c, as. Let X,( t ) = MXt,(t)). Show that X , , . . . , X , are orthogonal IF',,,,}-martingales. More generally, show that for any I c { 1,. . . , m}, n, I X , is an {.Ft,,,}-rnartingaIe.

20. Let 1 be a metric lattice. Show that a filtration { f , , u E 9) is right continuous if and only if for every u, {u,) c f with u 5 u,, n = 1, 2,. . . , and u = lim,+,., u,, we have Pu = nSaF,.

21. (a) Suppose M is a local martingale and ~up,~,IM(s)( E t' for each t > 0. Show that M is a martingale.

(b) Suppose M is a positive local martingale, Show that M is a supermartingale.

22. Let X and Y be measurable and {q,}-adapted. Suppose €[IX(t)l Po 1 Y(s)l ds] < 00 and E[fo IX(s)Y(s)l ds] < m for every 2 0. and that X is a (Q,)-martingale. Show that X(r)fo Y(s)ds -Yo X(s)Y(s) ds is a martingale. (Cf. Proposition 3.2 but note we are not assuming X is right continuous.)

10. NOTES 93

Let X be a real-valued {$,)-adapted process, with E [ [ X ( t ) ( ] < OD for every I 2 0. Show that X is a {9,)-martingale if and only if E[X( r ) ] = E[X(O)] for every {Y,)-stopping time 'c assuming only finitely many values.

Let M,, . . . , M, be right continuous ('3,)-martingales, and suppose that, for each I c { I , . . . , n} , n16 I M, is also a {YJ-martingale. Let t l . . . . , 'cn be {Y,}-stopping times, and suppose €[nl=I supfst , lM~r)l] < OD. Show that M(r) I n;=, M,(t A T,) is a {YJ-martingale. Hint: Use Problem 23 and induction on n.

23.

24.

25.

26.

27.

Let X be a real-valued stochastic process, and (9,) a filtration. ( X is not necessarily {F,)-adapted.) Suppose E [ X ( t ) ( S , ] 2 0 for each t . Show that E [ X ( r ) l F , ] 2 0 for each finite, discrete {9,}-stopping time 'c.

Let (M, A, p) be a probability space, and let A, c Az c . . . be an increasing sequence of discrete a-algebras, that is, for n = 1, 2,. . . , A, = a(A;, i = 1, 2,. . .) where the A; are disjoint, and M = U , A ; . Let X E L'(p), and define

(9.17)

(a) Show that ( X , ) is an martingale. (b) Suppose A = v,, An. Show that

Let ( X ( t ) : t E 3) be a stochastic process. Show that

X , = X pa.s. and in L'(,u).

(9.18) o(X(s): s E f) = u a(X(s): s E I ) I C J

where the union is over all countable subsets of $.

28. Let T and u be {F,}-stopping times. Show that S,,, = Sf n 9. and 9,,, = F, V F , .

29. Let X be a right continuous, E-valued process adapted to a filtration {S,}. LetfE C(E) and g, h E B(E), and suppose that

(9.19)

and rt


are {FJ-martingales. Show that

is an {s,)-martingale.

10. NOTES

Most of the material in Section 1 is from Doob (1953) and Dynkin (1961), and has been developed and refined by the Strasbourg school. For the most recent presentation see Dellacherie and Meyer (1978, 1982). Section 2 is almost entirely from Doob (1953).

The notion of a local martingale is due to It8 and Watanabe (1965). Propo- sition 3.2 is of course a special case of much more general results on stochastic integrals. See Dellacherie and Meyer (1982) and Chapter 5. Proposition 3.4 is due to Doleans-Dade (1969). The projection theorem is due to Meyer (1968). Theorem 5.1 is also due to Meyer. See Meyer (1966), page 122,

The semigroup of conditioned shifts appeared first in work of Rishel(l970). His approach is illustrated by Problem 18. The presentation in Section 7 is essentially that of Kurtz (1975). Chow (1960) gave one version of an optional sampling theorem for martingales indexed by directed sets. Section 8 follows Kurtz (1980b).

Problem 4(b) is essentially Theorem 11.2.6 of Doob (1953). See Dellacherie and Meyer (1978), page 99, for a more refined version.

3

In this chapter we study convergence of sequences of probability measures defined on the Borel subsets of a metric space (S. d) and in particular of D,[O, a), the space of right continuous functions from [O, 00) into a metric space (E, r ) having left limits. Our starting point in Section I is the Prohorov metric p on 9(S) , the set of Borel probability measures on S, and in Section 2 we give Prohorov's characterization of the compact subsets of SyS). in Scction 3 we define weak convergence of a sequence in 9 ( S ) and consider its relationship to convergence in the Prohorov metric (they are equivalent if S is separable). Section 4 concerns the concepts of separating and convergence determining classes of bounded continuous functions on S.

Sections 5 and 6 are devoted to a study of the space Dc[O,co) with the Skorohod topology and Section 7 to weak convergence of sequences in P(DEIO, m)). In Section 8 we give necessary and suflicient conditions in terms of conditional expectations of r8(X,(c + u), X,(r)) A 1 (conditioning on S,".) for a family of processes {X , ) to be relatively compact (that is, for the family of distributions on DEIO, GO) to be relatively compacl). Criteria for relative compactness that are particularly useful in the study of Markov processes are given in Section 9. Finally, Section 10 contains necessary and sufficient conditions for a limiting process to have sample paths in C,[O, 00).

95

CONVERGENCE O F PROBABILITY MEASURES



1. THE PROHOIOV MLTRlC

Throughout Sections 1 4 . (S, d) is a metric space (d denoting the metric), @(S) is the a-algebra of Borel subsets of S, and N S ) is the family of Borel probability measures on S. We topologize WS) with the Prohorov metric

(1.1) where Y is the collection of closed subsets of S and

p(P, Q) = inf { E > 0: P(F) 5 Q(Fc) + E for all F E (B),

F' = x E S: inf d(x, y ) < E .

To see that p is a metric, we need the following lemma. { y 8 F I (1.2)

1.1 Lemma Let P, Q E 9 ( S ) and u, f l > 0. If

(1.3)

( 1.4)

W) 5 Q(Fa) + P

Q(F) s P(Fa) + B

for all F E W, then

for all F E W.

Proof. Given FI E '6, let F, = S - f l , and note that F2 E '6 and F, c S - q. Consequently, by (1.3) with F = F 2 ,

(1.5) implying (1.4) with F = F 1 .

OF;) = 1 - P(F2) 2 1 - Q(F9 - B 2 Q ( F , ) - P, CI

It follows immediately from Lemma 1.1 that p(P, Q) = p(Q, P) for all P, Q E qS) . Also, if p(P, Q ) = 0, then P(F) = Q(F) for all F E '6 and hence for all F E g(S); therefore, p(P, Q ) = 0 if and only if P = Q. Finally, if P, Q, R t SYS), p(P, Q) < 4 and AQ, R ) < E, then

( 1.6) P(F) < Q(F? + S < Q(F) + 6 S R((FY) + 6 + E 5 R(F'+*) + S + E

for all F E W, so p ( f , R ) s S + E, proving the triangle inequality.

rov metric when S is separable. The following theorem provides a probabilistic interpretation of the Proho-

1.2 Theorem Let (S, d ) be separable, and let P, Q E P(S). Define d ( P , Q) to be the set of all p E SyS x S) with marginals f and Q (i.e., p(A x S) = P(A) and p(S x A) = Q(A) for all A E @(S)). Then

(1.7) p(P, Q) = inf c 4 p . 0)

inf {e > 0: pc((x, y) : d(x, y ) 2 E ) 5 E } .

1. WE PROHOROV MnRlC 97

Proof. If for some E > 0 and p E M P , Q) we have

(1.8) p{(x , y ) : 4 x . Y ) 2 6 ) s 6,

then

( 1.9) P ( 0 = p(F x S)

5 P((F x S) n {(x, y ) : 4x9 Y ) < 4 ) + &

5 p(S x P) + E = Q(F') + E

for all F E W, so p(P, Q) is less than or equal to the right side of(1.7). The reverse inequality is an immediate consequence of the following lemma,

0

1.3 lemma Let S be separable. Let P, Q E B(S), p(P, Q) < 6, and 6 > 0. Suppose that E l . . . . , EN E $i?(S) are disjoint with diameters less than 6 and that P(E,) 5 6, where E, = S - uT= ,I&. Then there exist constants c,,. . ., cN E [O, I ] and independent random variables X , Yo,. . . , YN (S-valued) and ( ( [ O , I]-valued) on some probability space (Q, 9, v ) such that X has distribution P, C: is uniformly distributed on [O, I],

1 4 on ( X ~ E , , t r c , } , i = I ,..., N,

has distribution Q,

(1.11) {d(X. Y) z 6 + E ) c { X E E,} u

and

(1.12) v{d(X, Y) 2 6 + E } I; 6 + E.

The proof of this lemma depends on another lemma.

1.4 lemma A, E a ( S ) for i = I , . . . , n. Suppose that

(1.13) 1 pi 5 p(;, A,) for all I c (1 . . . . , n) .

Then there exist positive Borel measures A , , . . . , I , on S such that IAA,) = A,@) = pi for i = I , . . . , n and I;=, 1,(A) s p(A) for all A E a(S).

Let p be a finite positive Borel measure on S, and let pi 2 0 and

i c l

Proof. Note first that it involves no loss of generality to assume that each

We proceed by induction on n. For n = I , define 1, on @(S) by A1(A) I pi > 0.

98 CONVERGENCE OF PROIMILIM W l J R E S

pIC((A n Al)/C((Al). Then JAAl) = A,@) = p, , and since pl s AA,) by (1.131, we have A,(A) s p(A n A , ) S p(A) for all A E @S). Suppose now that the lemma holds with n replaced by m for m = I , . . . , n - 1 and that p, pi, and A, (1 S i 5 n) satisfy (1.13). Define q on &(S) by q(A) = p(A n A,)/p(A,), and let E, be the largest E such that

(1.14) forall I c { I , ..., n - I } . I P I

CASE 1. E, L p.. Let 1, - p.q and put p' = p - A,. Since p . s p(A.) by (1.13), p' is a positive Borel measure on S, so by (1.14) (with E = p,) and the induction hypothesis, there exist positive Borel measures A,, . . . , A,,-, on S such that l , (A i ) = 1,(S) = pi for i = 1,. . . , n - I and I;:,' L,(A) 5 $(A) for all A s 9(S). Also, A,@,) = A,@) = p,,, so A,, . . . , 1, have the required properties.

CASE 2. E, < pm. Put p' = p - ~ , q , and note that p' is a positive Borel measure on S. By the definition of E,, there exists I , c ( I , . . .,n - 1) (nonempt y) such that

with equality holding for I = I o . By the induction hypothesis, there exist positive Borel measures Ai on S, i E I , , such that AAA,) = 11s) = p , for each i f I . and zlc l o AXA) 5 PYA) for all A E @S). Let p; = p , for i = I , . . . , n - 1 and p l = p, - e 0 . Put Bo = U , . I O A i , define p" on g(S) by p"(A) = p'(A) - p'(A n Bo), and let I , = ( I ,..., n) - I,. Then, for all I c I,,

Here, equality in the fint line holds because equality in (1.15) holds for I -- I,, while the inequality in the second line follows from (1.14) if n 9 I and from (1.13) if n E I; more specifically, if n E I, then

(1.17)

1. THE PROHOROV M R l C 99

By (1.161,

(1.18) c P; e&J/ A,) for all I c 11, i r I

so by the induction hypothesis, there exist positive Borel measures A; on S, i e l I , such that &(Ai) = &(S) = pi for each i~ I , and ~ i G l l X ~ A ) 5 $‘(A) for all A E a(S) . Finally, let A, = A; for i e I , - ( n ) and A, = A; + con. Then AAA,) = Ai(S) = pi for each i E I,, hence for i = 1,. . . , n, and

(1.19) i - I i e lo l C l l

= 1 A,(A n B,) + C R;(A) + E , ) ~ ( A )

s PYA n B,) + p”(A) + eOrt(A)

= PYA) + Eo M A )

= P ( 4

for all A E .Sa(S), so A,, . . . , A n again have the required properties.

i r l o i C l l

0

1.5 Corollary and A, E a ( S ) for i = 1, . . . , n. Let E > 0, and suppose that

Let p be a finite positive Borel measure on S, and let p , 2 0

p i S p u A, + E for all I c { I , ..., n } . L r I ) ( I .20) i C l

Then there exist positive Borel measures A , , . . . , A n on S such that A,(A,) = A A S ) s p i for i = 1, ..., n, ~ ; = l A , ( S ) r ~ ; , l p , - ~ , and C ; = l A , ( A ) s p ( A ) for all A E 9?(S).

Proof. Let S’ = S u {A}, where A is an isolated point not belonging to S, Extend c( to a Borel measure on S’ by defining p((A}) = E. Letting A; = A, u {A} for i = 1,. , . , n, we have

(1.21)

By Lemma 1.4, there exist positive Borel measures A’,, . . . ,A ; on S such that U A ; ) = Al(S’) = pi for i = 1,. . . , n and CY- I AKA) 5 p(A) for all A E 9?(S). Let Ai be the restriction of 2; to .Sa(S) for i = 1,. . . ,n . Then Ai(Ai) = &(A,) 5 A;(A;) = p i and A,(S - A,) = A;(S - A;) = 0 for i = I , . . . , n. Also,

100 CONVERGENCE OF PROMIlUTY MEASURES

Proof of lemma 1.3 Let P, Q, &,a, and E,, . . . , EN be as in the statement of the lemma. Let p, = P(E,) and A, = E f for i = I , . . . , N. Then

(1.23) Z p f S P + e forall I c { l , ..., Nj, 1 6 1

so by Corollary 1.5, there exist positive Bore1 measures A, , . . . , AN on S such that AAA,) = A@) I; pf for i = 1,. . . , N,

(1.24) I = I I - I

and cya, AAA) S Q(A) for all A cd?(S). Define cI, ..., cN E [O, 13 by c, = (p, - A,(S))/p,, where 0/0 = 0, and note that (1 - c,)P(EI) = A,(S) for i = I , . . . , N and f (Eo) + crN.p, cfP(E,) = 1 - zrNIl A,@). Consequently, there exist Qo, . . . , QN E 9(S) such that

(1.25)

and Q,(B)(l - c,)P(E,) = Ads), i = 1,. . . , N,

N N

I = 1 i= 1 (1.26)

for all B E 4?(S). Let X, Y o , . . . , YN, and t be independent random variables on some prob-

ability space (Q 9, v ) with X, 6,. . . , YN having distributions P, Q o , . . . , Q N and t uniformly distributed on [O, 13. We can assume that Yl,. . . , YN take values in A,, . . . , A N , respectively. Defining Y by ( l , lO), we have by (1.25) and

Qd&( P ( E d + ci 4EJ) = Q(B) - c JAB)

( 1 m, (1.27)

N

V { Y E B ) = C Qi(BM1 - C P ' O i ) i = 1

4 +PdB)(P(Ed+ I - c 1 CIP(E

N

= Q(4 for all B 8 5o(S). Noting that {X E: E l , < 2 cf} c (X E E l , Y E A f } c (d(X, Y) e 6 + E } for i = I , . , . , N, we have

N (1.28) {d(X, Y) 2 6 -t E } c {X E Eo} u u (X B E l , C: < c,}

I - 1

I. THE mottoaov m i c 101

where the third containment follows from p , - AXS) 5; E for i = 1,. .., N (see (1.24)). Finally, by the first containment in (1.28) and by (1.24),

(1.29) N

v ( d ( X , Y) 2 8 + E } 5; P(E,) + c iP(E, ) I - I

N

= P(E0) + c (Pi - W)) i = l

S d + E . 0

1.6 Corollary Let (S, d) be separable. Suppose that X , , n = I , 2,. . . , and X are S-valued random variables defined on the same probability space with distributions P,, n = I, 2,. . . , and P, respectively. If d(X, , X)-+ 0 in probability as n -+ 00, then limn-.s,p(Pm, P) = 0.

Proof. For n = 1, 2,. . . , let p, be the joint distribution of X , and X . Then lim,,,p,{(x, y) : d(x, y) 2 E } = 0 for every E > 0, so the result follows from Theorem 1.2. 0

The next result shows that the metric space (P(S), p) is complete and separable whenever (S, d) is. We note that while separability is a topological property, completeness is a property of the metric.

1.7 Theorem complete, then (B(S), p) is complete.

If S is separable, then 9 ( S ) is separable. If in addition (S, d) is

Proof. Let {x"} be a countable dense subset of S, and let 6, denote the element of P(S) with unit mass at x E S. We leave it to the reader to show that the probability measures of the form cr= a, S,, with N finite, a, rational, and cr= I ai = I, comprise a dense subset of B(S) (Problem 3). To prove completeness it is enough to consider sequences {P"} c 4yS) with

p(P,- P") < 2-" for each n 2 2. For n = 2, 3,. . . , choose E?',. . . , E$i E mS) disjoint with diameters less than 2-" and with P,- ,(lit') s 2-", where lib"' = S - urz1 EY1. By Lemma 1.3, there exists a probability space (Q, 9, v) on

which are defined S-valued random variables Yf', . , . , YE!, n = 2, 3 , . . . , LO, I]- valued random variables <'"', n = 2, 3, . . . , and an S-valued random variable XI with distribution PI , all of which are independent, such that if the constants .',"', . . . , CE E LO, 1 ], n = 2, 3 , . . . , are appropriately chosen, then the random variable

102 CONVERGENCE OF f'ROIAMLlM MEASURES

has distribution P, and (1.31)

successively for n = 2,3,. . . . By the Borel-Cantelli lemma, V { d ( X , - I , X,) 2 2-,+') 5 2--"+',

(1.32)

so by the completeness of (S, d), limn-mX, = X exists as. Letting P be the distribution of A', Corollary 1.6 implies that limn*m p(P,, P) = 0. a

As a further application.of Lemma 1.3, we derive the so-called Skorohod represent a tion.

1.8 Theorem Let (S ,d) be separable. Suppose P,, n = 1, 2,. . . , and P in 9 ( S ) satisfy limn-,m p(P,, P) = 0. Then there exists a probability space (Q, 9, w ) on which are defined S-valued random variables X,, n = I , 2,. . . , and X with distributions P,, n = I , 2,. . ., and P, respectively, such that limnem X, = X as.

Proof. For k = 1, 2,. . . , choose E','), . . , , E$i E wS) disjoint with diameters less than 2-' and with P(l$) s 2-', where li$' = S - U;lll$), and assume (without loss of generality) that && E minIs,,,P(Ej") > 0. Define the sequence {k,) by

( I .33)

and apply Lemma 1.3 with Q = P,, e = ek,/k, if k, > I and E = p(P,,, P ) + I/n if k, = 1, 6 = 2-'", El = and N = N, for n = 1, 2,. . , . We conclude that there exists a probability space @,S, v ) on which are defined S-valued random variables Yg),. . . , Y& n = 1, 2,. . . , a random variable < uniformly distributed on [O, 11, and an S-valued random variable X wifh distribution P, all of which are independent, such that if the constants c',"), . . . , cK! E [O, 11, n = 1,2,. . . , are appropriately chosen, then the random variable

has distribution P,, and

(1 3)

2. ?ROHOROV’S THEOREM 103

for n = 1, 2,. . . . If K,, E min,,, k, > 1, then

00

s c V { X € E $ ’ } + V ( < - k = K . i n ]

and since limn-.m K, = 00, we have limn-m X,, = X a.s. 0

We conclude this section by proving the continuous mapping theorem.

1.9 Corollary Let (S, d) and (S’, d’) be separable metric spaces, and let h: S4S’ be Borel measurable. Suppose that P,,, n = I , 2,.. ., and P in 9 ( S ) satisfy limn-00 p ( P , , P) = 0, and define Q, , n = I , 2,. . . , and Q in 9(S) by

(1.37) Q,, = P,h- ’ , Q = Ph- ’ .

(By definition, Ph-’(B) = P{s E S : h(s) E B}.) Let ch be the set of points of S at which h is continuous. If f(C,) = 1, then limn-mp’(Q,,, Q) = 0, where p‘ is the Prohorov metric on 9(S’).

Proof. By Theorem 1.8, there exists a probability space (a, 9, v ) on which are defined S-valued random variables X , , n = 1, 2,. . . , and X with distributions P,,. n = I , 2,. . . , and P, respectively, such that limn*mXn = X a.s. Since v { X E c h f = I, we have h(X,) = h(X) a.s., and by Corollary 1.6, this implies that p’(Qn, Q) = 0. 0

2. PROHOROV’S THEOREM

We are primarily interested in the convergence of sequences of Borel probability measures on the metric space (S, d). A common approach for verifying the convergence of a sequence {x,} in a metric space is to first show that (x,,) is contained in some compact set and then to show that every convergent subsequence of (x,) must converge to the same element x. This then implies that

x,, = x. We use this argument repeatedly in what follows, and, consequently, a characterization of the compact subsets of 9 ( S ) is crucial. This characterization is given by the theorem of Prohorov that relates compactness to the notion of tightness.

A probability measure P E 9 ( S ) is said to be tight if for each E > 0 there exists a compact set K c S such that P ( K ) L 1 - E. A family of probability measures .M c 9(S) is right if for each E > 0 there exists a compact set K c S

such that

inf f ( K ) 2 1 - e. P S . U

2.1 Lemma If (S, d) is complete and separable, then each P E 9(S) is tight.

Proof. Let {xk} be dense in S, and let P E HS) . Given E > 0, choose positive integers N,, N3 ,. . . such that

E p ( k c l B(Xb i)) 2 I - - 2"

for n = I , 2,. . . . Let K be the closure of nnz I u:: , B(x,, l/n). Then K is totally bounded and hence compact, and

2.2 Theorem Let (S, d) be complete and separable, and let & c 9(S) . Then the following are equivalent:

B A is tight. fb) For each E > 0, there exists a compact set K c S such that

(2.4) inf P(K') 2 1 - E. P a . 4

where K' is as in (1.2). (c) .& is relatively compact.

Proof. (a * b) Immediate. (b 9 c) Since the closure of A is complete by Theorem 1.7, it is SUE-

cient to show that .& is totally bounded. That is, given S > 0, we must construct a finite set M c B(S) such that .# c {Q: p(P, Q) < 6 for some

Let 0 < E < 6/2 and choose a compact set K c S such that (2.4) holds. By the compactness of K, there exists a finite set ( x I , . . . , x,) c K such that K' c u;-, B, , where Bi = E(x,, 28). Fix xo E S and m 2 n/E, and let Jf be the collection of probability measures of the form

P E Mj.

where 0 5 k, 5 m and c;=,, k, = m. Given Q E A, let k, = [mQ(E,)] for i = I ,..., n, where E, = 8,

1. -0HOROV'S THEOREM 105

- u;: S,, and let ko = m - c;i I k,. Then, defining P by (2.5), we have

for all closed sets F c S, so p(P, Q) 5 2~ < 6.

(c 3 a) Let E > 0. Since is totally bounded, there exists for n = I , 2,. . . a finite subset N, c AY such that d c {Q: p ( P , Q) < . ~ / 2 " ' ~ for some P E N,}. Lemma 2.1 implies that for n = I , 2,. . . we can choose a compact set K, c S such that P ( K , ) z 1 - ~ / 2 " " for all P E Jv,. Given Q E ..M, it follows that for n = 1,2,. . . there exists P, E N, such that

Letting K be the closure of f l , 2 1 K ~ ' 2 n i ' , we conclude that K is compact and

n

2.3 Corollary A is rela'tively compact.

Let (S, d) be arbitrary, and let A' c *S). If A is tight, then

Proof. For each m 2 I there exists a compact set Km c S such that

(2.9) 1 inf P(K,) 2 1 - -,

P * . M m

and we can assume that K , c K, c * . For every P E d and m L 1, define I"m) E SyS) by P'")(A) = P(A n K,) / f (K, ) . and note that I"'"' may be regarded as belonging to 9 ( K m ) . Since compact metric spaces are complete and separable, M'") = {Pen': P E ,rV) is relatively compact in cP(K,) for each m 2 1 by Theorem 2.2.

106 CONVERGENCE OF PROBABILITY MEASURES

We also have

(2.12) P(A) 2 P(K,)P'"''(A) 2 (1 - -!)P'"yA), m

for all P E A, A E 4?(S), and m 2 1. By (2.101,

I m

(2.14)

for all P E d a n d m T 1 . Given A , , A , , ... ~~(S)d i s jo in t , (2 .13 )and (2.11) imply that

(2.15)

p(f, P'")) s -

G I P'"'(Ai) - P ( A i ) 1 I

2 2 4 m

s;m+;=-

for every P E d and m' > m 2 1. Let {fm} c A. By the relative compactness of &"' in flK,,,), there exists

(through a diagonalization argument) a subsequence {P,,,} c {P,,} and Q'"') E P(K,,,) such that

(2.16)

for every m 2 1. It follows that

(2.17)

for all closed sets F c S and m 2 1, and therefore the inequalities (2.11) and (2.13) are preserved for the QIm) for all closed sets A c S, hence for all A E WS) (using the regularity of the Q'")). Consequently. we have (2.15) for the Q("'), so

(2.18) Q(A) s lim Q("'(A) m-OD

3. WEAK CONVERGENCE 107

exists for every A E a(S) and defines a probability measure Q tz @(S) (the countable additivity following from (2.1 5)). But

(2.19) P(Pn,* Q) 5 P(Pn,, PP') + P(C:'~ Q""') + P(Q'"', Q)

for each k and m 2 1, implying that limk-,m AP,,,. Q) = 0. 0

We conclude this section with a result concerning countably infinite product spaces.

2.4 Proposition Let (Sk, dk), k = I , 2,.. ., be metric spaces, and define the metric space (S, d ) by letting S = nL"- Sk and d(x, y) = z?= I 2-'(dk(x,, y k ) A 1 ) for all x, y E S. Let { P a } c P(S) (where a ranges over some index set), and for k = 1 , 2 , . . . and each a, define Pd E p(&) to be the kth marginal distribution of P, (i.e., p". = Pan; ' , where the projection 1 [ k : S-r Sk is given by q (x ) = xk). Then {Pa} is tight if and only if (ft} is tight for k = I , 2,. . . .

Proof. Suppose that {e} is tight for k = 1, 2 , . . . , and let E > 0. For k = I, 2 , . . . , choose a compact set Kk c S k such that inf,Pt(Kk) 2 I - 42'. Then K = np= K, = ()?= I n; I(&) is compact in S, and

(2.20)

for all a. Consequently, {P,} is tight.

compact in S, and The converse follows by observing that for each compact set K c S, q ( K ) is

(2.21)

fork= 1,2, . . . .

inf Pt(nk(K)) 2 inf P,,(K) (I U

0

3. WEAK CONVERGENCE

Let C(S) be the space of real-valued bounded continuous functions on the metric space (S, d ) with norm 11/11 = sup,,,I/(x)l. A sequence {P,,} c g(S) is said to converge weakly to P E @(S) if

lim f dP, = J dP, f E C(S).

The distribution of an S-valued random variable X, denoted by P X - I , is the element of SyS) given by P X - ' ( B ) = P { X E B ) . A sequence { X , } of S-valued

(3.1) n-m s 5

108 CONVERGENCE OF PROMIlLlfV MEASURES

random variables is said to conuerge in distribution to the S-valued random variable X if {PX;') converges weakly to PX-', or equivalently, if

(3.2)

Weak convergence is denoted by P, * P and convergence in distribution by X , * X . When it is useful to emphasize which metric space is involved, wc write P, =+ P on S" or "X,, =+ X in S".

If S' is a second metric space and/: S-+ S' is continuous, we note that then X, - X in S implies f ( X , ) =./(X) in S since g E c(S) implies g 0 /E c(S). For example, if S = C[O, 13 and S' = W, thenf(x) 3 supo,,, x(r) is continuous, so X , * X in C[O, I ] implies S U ~ ~ ~ , ~ X&)* S U ~ ~ ~ , , X( t ) in R. Recall that, if S = R, then (3.2) is equivalent to

Iim ECf(xJI = ECf(X)I, n-oo

JE CCS,.

(3.3) lim P { X , r; x} = PIX s x} n - m

for all x at which the right side of (3.3) is continuous. We now show that weak convergence is equivalent to convergence in the

Prohorov metric. The boundary of a subset A t S is given by dA = A' n ( A and A' denote the closure and complement of A, respectively). A is said to be a P -continuity set if A e 9 ( S ) and P(dA) = 0.

3.1 Theorem Let (S, d ) be arbitrary, and let {P,,} c f i S ) and P E 9(S) . Of the following conditions, (b) through (f) are equivalent and are implied by (a). If S is separable, then all six conditions are equivalent:

lim,,-.mp(Pn, P) = 0.

Iim,,-- j'JdP, = f dP for all uniformly continuousfc c(S). limn-- P,,(F) 5 P(F) for all closed sets F c S.

P,(G) 2 P(G) for all open sets G c S. limn-* PdA) = P(A) for all P-continuity sets A c S.

P, 9 P.

-

3. WEAY CONVERGENCE 109

Consequently,

- lim J (II/II +/)dPn 5 J (II/II + / ) d ~ , n-m

(3.6)

iiii ~ ' ( I I ~ I I -ndPa (II/II - n d p a-+ m I

for allfe e(S), and this implies (3.1).

(b * c) Immediate. (c =5 d) Let F c S be closed. For each E > 0, define/, tz c ( S ) by

(3.7)

where d(x, F) = i d y e Fd(x , y). Thenfis uniformly continuous, so

(3.8)

for each E > 0, and therefore

lim Pa(F) 5 lirn $, dPa = 1; d P , I I I

- n-m n - * ~

lirn Pn(F) s lim f , dP = P(F).

(d el For every open set G c S,

- a-tw 8 - 0

(3.9)

- (3.10) - lirn Pa(G) = I - lirn fa(G') 2 I - f(G') = P(G).

a - Q a- w

(c - f) Let A be a P-continuity set in S, and let A" denote its interior (A" = A - aA) . Then

(3.1 I ) tim Pn(A) s lim PdJ) = 1 - c lirn PA,$) s 1 - P($) = P(A)

and

(3.12) - lirn Pn(A) Lm P,,(A") 2 P(A") = P(A).

a-m n-+m n+m

a-m a- w

(f-b) Let /€ C(S) w i t h J r 0 . Then d { f r t ) c { j = t ) , so (fr r } is a P-continuity set for all but at most countably many t 2 0. Therefore,

(3.13) lim 1 f dP, = !:I [ " ' P a { / > t } d r

I1 1 II

n-m

= P { f r t } dt = s / d P

for all nonnegative/€ C(S), which clearly implies (3.1).

110 CONVERGENCE OF PRODAUUN MLASMES

(c =a, assuming separability) Let E > 0 be arbitrary, and let E l , E , , . . . E g ( S ) be a partition of S with diameter(E,) < ~ / 2 for i = 1, 2,. . . . Let N be the smallest positive integer n such that P(u;sl E,) > I - ~ /2 , and let Y be the (finite) collection of open sets of the form ( U , c l E , ) 1 1 2 , where I c { 1,. . . , N}. Since Y is finite, there exists no such that P(G) s P,(G) + ~ / 2 f o r a l l G E Y a n d n r n o . G i v e n F E ( 8 1 e t

(3.14) F , = u { E i : 1 S i -< N, El A F # fa).

Then FZ2 E Y and

(3.15) P( F ) s P( Ffl') + ~ / 2

s Pa( F y ) + E 5 Pa( Fa) + E

for all n 2 no. Hence p( Pa, P) s E for each n 2 no. 0

3.2 Corollary Let Pa, n = I, 2,. . . , and P belong to qS), and let S' E A?(S). For n = 1 , 2,. . ., suppose that P,(S') = P(S') = 1, and let P: and P' be the restrictions of Pa and P to @(S) (of course, S' has the relative topology). Then Pa - P on S if and only if Pi =a P an S'.

Proof. If G' is open in S', then G' = G n S' for some open set G c S. There- fore, if Pm =+ P on S,

(3.16) - lim P:(G) = Lm P,(G) 2 P(G) = P(G'), a--m a-oo

so PL - P on S' by Theorem 3, I. The converse i s proved similarly. D

3.3 Corollary Let (S, d) be arbitrary, and let (X", 9, n = I , 2,. . . , and X be (S x S)- and S-valued random variables. If X, + X and d ( X , , G)-+ 0 in probability, then U, X.

3.4 Remark If S is separable, then @(S x S) = 9 ( S ) x i3(S), and hence ( X , Y) is an (S x S)-valued random variable whenever X and Y are S-valued random variables defined on the same probability space. This observation has already been used implicitly in Section 1, and we use it again (without

0 mention) in later sections of this chapter.

Proof. I f f € C(S) is uniformly continuous, then

(3.17) lim Eu(X , ) -f(Y,,)] = 0. a-+m

Consequently,

(3.18)

and Theorem 3.1 is again applicable.

lim ET/(U,)l = lim Ec/(Xa)] = EU(W1, n-m n-m

0

4. SEPARATING AND CONVERGENCE DETERMINING SETS 111

4. SEPARATING AND CONVERGENCE DETERMINING SETS

Let (S, d ) be a metric space. A sequence IS,) c B(S) is said to converge boundedly and pointwise to / E B(S) if sup, JIf, II < 00 (where 11 . II denotes the sup norm) and limn+mfn(x) = f ( x ) for every x E S; we denote this by

bp-lim f , =f n - m

A set M c B(S) is called bp-closed if whenever {jn} c M. /E gS), and (4.1) holds, we have /E M . The hp-closure of M c B(S) is the smallest bp-closed subset of B(S) that contains M. Finally, if the bp-closure of M c B(S) is equal to B(S), we say that M is bp-dense in B(S). We remark that if M is bp-dense in B(S) and f E B(S). there need not exist a sequence {h} c M such that (4.1) holds.

4.1 lemma su bspace.

If M c B(S) is a subspace, then the bp-closure of M is also a

Proof. Let H be the bp-closure of M. For eachfe H, define

(4.2)

and note that H , is bp-closed because H is. I f f€ M, then H, 3 M, so H , = H. I f f E H, then f E H, for every g E M, hence g E H, for every g E M, and

H , = {g E H : af + bg E H for all a, b E Oa},

therefore H , = H. 0

4.2 Proposition Let (S, d) be arbitrary. Then e(S) is bp-dense in B(S). If S is separable, then there exists a sequence {f"} of nonnegative functions in C(S) such that span {I.} is bp-dense in B(S).

Proof. Let H be the bp-closure of c(S). H is closed under monotone convergence of uniformly bounded sequences, H is a subspace of E(S) by Lemma 4.1, and zG E H for every open set G c S. By the Dynkin class theorem for functions (Theorem 4.3 of the Appendixes), H = B(S).

If S is separable, let { x i } be dense in S. For every open set G c S that is a finite intersection of !?(xi, I/&), i, k L 1, choose a sequence (If) of nonnegative functions in 4 s ) such that bp-lim,+,f~ = x G . The Dynkin class theorem for

0 functions now applies to span { f t : n, G as above}.

For future reference, we extend two of the definitions given at the beginning of this section. A set M c B(S) x B(S) is called bp-closed if whenever {CJn. 9,)) c M, (f, 8) e 4s) x &S), bp-limn-mfw =1; and bp-lim,+,g, = g, we have (f, g) E M. The bp-closure of M c B(S) x B(S) is the smallest bp-closed subset of E(S) x E(S) that contains M.

112 CONVERCENCE OF nowtun MEASURES

A set M c 4s) is called separating if whenever P, Q E P(S) and

(4.3) I / d P = J f d Q , J E M,

we have P = Q. Also, M is called conoergence determining if whenever { P,,} c SyS), P E 9 (S) , and

(4.4)

we have P,, OD P. f dP = f dQ is

bp-closed. Consequently, Proposition 4.2 implies that c(S) is itself separating. It follows that if M c g(S) is convergence determining, then M is separating. The converse is false in general, as Problem 8 indicates. However, if S is compact, then 9 ( S ) is compact by Theorem 2.2, and the following lemma implies that the two concepts arc equivalent.

Gives P, Q E 4yS), the set of all fc B(S) such that

4.3 Lemma Let {P,,} c 9 ( S ) be relatively cornpact, let P E as), and let M c c(S) be separating. If (4.4) holds, then P,, =+ P.

Proof. If Q is the weak limit of a convergent subsequence of {P,,), then (4.4) 0 implies (4.3), so Q = P. it follows that P,, * P.

4.4 Proposition Let (S, d ) be separable. The space of functionsfE c(S) that are uniformly continuous and have bounded support is convergence determining. If S is also locally compact, then C,(S), the space of/€ c(S) with compact support, is convergenca determining.

Proof. Let {x,} be dense in S, and defineh, E c(S) for is j = 1,2,. . . by

(4.5) s,I(x) = 2( 1 - jd(x, xi)) V 0.

Given an open set G c S, define g,,, E M for m = I, 2,. . . by g,,,(x) = (xh,(x)) A 1, where the sum extends over those i, j s m such that B(x,, l/j) c G (and ax,, 10) is compact if S is locally compact). If (4.4) holds, then

- lim P,(G) 2 lim /a,,, dP,, = [gm dP (4.6)

for m = 1, Z..,, so by letting m+ 00, we conclude that condition (e) of Theorem 3.1 holds. 0

r - r m n-r m

Recall that a collection of functions M c c(S) is said to separate points if for every x, y E S with x # y there exists h E M such that h(x) # Yy). In

4. SRAMTINC AND CONVERGENCE DFl€IMMINc SETS 113

addition, M is said to strongly separate points if for every x E S and S > 0 there exists a finite set { h , , . . . , h k ) c M such that

(4.7)

Clearly, if M strongly separates points, then M separates points.

4.5 Theorem algebra.

Let ( S , d ) be complete and separable, and let M c c(S) be an

(a) If M separates points, then M is separating. (b) If M strongly separates points, then M is convergence determining.

Proof. (a) Let P, Q E qS), and suppose that (4.3) holds. Then h dP = h dQ for all h in the algebra H = {f+ a:/€ M , a E R}, hence

for all h in the closure (with respect to 11 * 11) of H. Let g E c(S) and E > 0 be arbitrary. By Lemma 2.1, there exists a compact set K c S such that P ( K ) 2 1 - E and Q ( K ) 2 I - E. By the Stone-Weierstrass theorem, there exists a sequence { g , } c H such that supxr Ig,(x) - g(x ) l -+ 0 as n--r OD. Now observe that

for each n, and the fourth term on the right is zero since g,e-'h' belongs to the closure of H. The second and sixth terms tend to zero as n -+ OD, so the left side of (4.8) is bounded by 4y&, where y = sup,., w-". Letting E-+ 0,

114 CONVERGENCE OF PROBMIUTV MUSURE

it follows that I g d P = fgdQ. Since g E 4s) was arbitrary and C(S) is separating, we conclude that P = Q.

(b) Let (P,} c P(S) and P E: *S), and suppose that (4.4) holds. By Lemma 4.3 and part (a), it suffices to show that { P,,) is relatively compact.

Let fl , . . . ,& E M. Then

lim g 0 uI,. . . ,j;)dPn = Jg 0 C T ~ , . . . , s , ) ~ P

for all polynomials g in k variables by (4.4) and the assumption that M is an algebra. Sincef,, . . . ,& are bounded, (4.9) holds for all g E C(0a')). We conclude that

n-m s (4.9)

(4.10) P n U l , . * . , j ; ) - ' ~ ~ , , . . . , ~ ) - ' , fl,-.-,/;, E M.

Let K c S be compact, and let S > 0. For each x E S, choose {hf, . . . , hi,,)} c M satisfying

(4.1 1) ~ ( x ) 3 inf max Ih;(y) - h;(x)I > 0,

and let G, = ( y E S: max,sisk~,)/h~(y) - h:(x)I < 4 x ) ) . Then K c u, Gx c Kd, so, since K is compact, there exist xl,. . . , x,,, E K such that K c u;"! I G,, c K*. Define g,, . . . , g, IZ c(S) by

y : l ( i . x ) ~ d I s i s & ( x )

(4.12)

and observe that (4.10) implies that

(4.14) - lim P,,(Kd) 2 &I P,, u G,, n+aD I - m

= lim Pn x E S: min [gl(x) - E(xI)] < 0

2 P x s: min [gxx) - &(XI)] < 01

I 1 s l s m - n-m

I l s l s m

= p ( G 1 Gx,)

2 P ( K h

where the middle inequality depends on (4.13) and Theorem 3.1. Applying Lemma 2.1 to P and to finitely many terms in the sequence {Pn}, we conclude that there exists a compact set K c S such that inf, P,,(K'J 2

0 1 - 6. By Theorem 2.2, { Pn) is relatively compact.

4. SEPARATING AND CONVERGENCE DRRMlNlNG W S 115

We turn next to a result concerning countably infinite product spaces. Let ( s k , dk), k = 1, 2 , . . . , be metric spaces, and define s = nCa s k and d(x, y) = x F = l 2-'(dk(Xk, yk)A I ) for all x, y e S. Then (S, d) is separable if the Sk are separable and complete if the (Sk , d,) are complete. If the Sk are separable, then B(S) = nFm I a(&).

(a) If the S, are separable and the M k are separating, then M is separat-

(b) If the (Sk , d k ) are complete and separable and the M k are con- ing.

vergence determining, then M is convergence determining.

Proof. (a) Suppose that P, Q E B(S) and

and let p 1 and v1 be the first rnarginals of p and v on i4#(Sl). Since MI is separating (with respect to Borel probability measures), it is separating with respect to finite signed Borel measures as well. Therefore p 1 = v 1 and hence

whenever A , E O ( S l ) , n ;r 2, andfk E M, u { I } for k = 2 , . . . , n. Proceeding inductively , we conclude that

whenever n 2 1 and A, E .4a(S,) for k = 1, . . . , n. It follows that P = Q on nF= , d?(S,) = B(S) and thus that M is separating.

(b) Let { P m } c B(S) and P E B(S), and suppose that (4.4) holds. Then, for k = I , 2, ..., j j d P ! = j j d P for aIlJc M k , where P! and P' denote the k th marginals of P,, and P, and hence P: =. P'. In particular, this implies that {Pi} is relatively compact for k = 1, 2,. .. , and hence, by

Theorem 2.2 and Proposition 2.4, {f,) is relatively compact. By Lemma 17 4.3, P, 3 P, so M is convergence determining.

We conclude this section by generalizing the concept of separating set. A set M c B(S) is called separating if whenever P, Q E 9(S) and (4.3) holds, we have P = Q. More generally, if -4 c P(S), a set M c M(S) (the space of real-valued Bore1 functions on S) is called separaring on .4 if

(4.20) 1111 dP K 00, / E M, P E -4.

and if whenever P, Q E

set of monomials on w (i.e., 1, x, x2, x', . . .) is separating on and (4.3) holds, we have P = Q. For example, the

(Feller (1971), p. 514).

Throughout the remaining sections of this chapter, (E, r) denotes a metric space, and q denotes the metric r A 1.

Most stochastic processes arising in applications have the property that they have right and left limits at each time point for almost every sample path. It has become conventional to assume that sample paths are actually right continuous when this can be done (as it usually can) without altering the finitedimensional distributions. Consequently, the spacc DEIO, 00) of right continuous functions x : [O, GO)+ E with left limits (ie., for each r 2 0, lim,,,, x(s) = x(t) and lim,,,, x(s) ~i x(r-) exists; by convention, lim,,o- x(s) = x(0-) = x(0)) is of considerable importance.

We begin by observing that functions in DEIO, a) are better behaved than might initially be suspected.

5.1 discon tin ui ty.

lemma If x E DEIO, a), then x has at most countably many points of

Proof. For n = 1, 2,. . . , let A, = (r > 0: r(x(t), x(r-)) > l/n}, and observe that A, has no limit points in [O, ao) since lim,,,, x(s) and Iim,-,- x(s) exist for all t 2 0. Consequently, each A, is countable. But the set of all discontinuities

0

The results on convergence of probability measures in Sections 1 4 are best suited for complete separable metric spaces. With this in mind we now define a metric on DEIO, 00) under which it is a separable metric space if E is separable,

of x is u."-, A,, and hence it too is countable.

5. THE SIACE D#, ac) 117

and is complete if ( E , r ) is complete. Let A' be the collection of (strictly) increasing functions A mapping [0, 00) onto [O, 00) (in particular, 1(0) = 0, lim14m A(c) = 00, and A is continuous). Let A be the set of Lipschitz continuous functions rl E A' such that

?(A) 3 ess sup I log X( t ) I t20

A(.$) - A(r )

For x, y E D,[O, 00). define

(5.2) d(x, y ) = inf [ y ( l ) V e-"d(x, y, 1, u) du , A s A 1

where

d(x, y, A, u) = SUP q(x(t A u), y(M) A u)). I 2 0

(5.3)

It follows that, given (x,,}, ( y,,} c DEIO, a), lima-md(xn, y,,) = 0 if and only if there exists (A,,} c A such that

(5.4)

y(rl,) = 0 and

lim m(u E [0, uo] : d(x, , y,, , A,,, u) 2 E } = 0 a - m

for every E > 0 and uo > 0, where m is Lebesgue measure; moreover, since

(5.5)

for every 1 E A,

implies that

(5.7)

lim HA,) = 0 n-m

lim sup I i a ( t ) - tl = 0 n-m O S I S T

for all T > 0. Let x, y E D,[O, a), and observe that

SUP q(x(t A u), y(A(t) A u)) = SUP 4(x(A- ' ( 1 ) A u), y(t A u))

for all 1 E A and u 2 0, and therefore d(x. y, 1, u ) = d(y, x, A-', u). Together with the fact that y(A) = y ( A - ' ) for every I 8 A, this implies that d(x, y) = d ( y , x). If d(x, y) = 0, then, by (5.4) and (5.7), x(r) = y( t ) for every

120 I 2 0 (5.8)

118 CONVERGENCE OF PRCMMIUTY MEASURES

continuity point t of y, and hence x = y by Lemma 5.1 and the right continuity of x and y. Thus, to show that d is a metric, we need only verify the triangle inequality. Let x, y, z E DJO, a), A1, I 2 E A, and u 2 0. Then

(5.9) SUP 4(Nt A u), Z ( ~ l ( A l ( 0 ) A 4) I20

5 SUP d x ( t A u), Y(J,(t) A 4) I20

+ SUP 4(Y(A,(t) A u), Z M A l ( f ) ) A 4) 120

= SUP q(x(t A u), v(.W)A 4) (LO

+ SUP dY(t A 4, z(.tz(t) A u)), rho

that is, d(x, z, A , 0 A, , u) s d(x, y, A , , u) + d(y, z, A,, u). But since 1, 0 1, E A and

(5.10) Y ( 1 , O 1,) r(4) + r(A2b

we obtain d(x, z) s d(x, y ) + d(y, 2).

topology. The topology induced on DEIO, a) by the metric d is called the Skorohod

5.2 Proposition Let {x,,} c DEIO, a) and x E Ds[O, 00). Then Iirnm-- d(x,, x) = 0 if and only if there exists {A,} c A such that (5.6) holds and

(5.1 1) lim d(x,, x, A,, u) = 0 for all continuity points u of x. n-rn

In particular, limm-ad(xm, x ) = 0 implies that limn-= x,(u) = limm~axn(u-) = x(u) for all continuity points u of x .

Proof. The sufliciency follows from Lemma 5.1. Conversely, suppose that lirnn4m d(x,, x ) = 0, and let u be a continuity point of x. Recalling (5.4). there exist {A,,} c A and {u,] c (u, a) such that (5.6) holds and

(5.12)

Now

lirn sup q(x,(t A u,), x(R,(t) A u,)) = 0. n-m 120

5. M W A a D,p, 00) 119

s SUP dxn(t A U n h x ( A n ( r ) A un))

4- SUP dx(s). x(4)

V SUP dx(A(u) A U n h Ns))

o s t s u

(I $ 3 S A d M ) v Y

Adu) n Y Sr su

for each n, where the second half of the second inequality follows by considering separately the cases r 5 u and t > u. Thus, limn-md(xn, x, A,, u) = 0 by

0 (5.12), (5.7). and the continuity of x at u.

5.3 Proposition Let (x,) c &LO, 00) and x E DEIO, GO). Then the following are equivalent:

(a) limndm d(x, , x) = 0.

(b) There exists (A,} c A such that (5.6) holds and

(5.14) lim SUP dxn(t)* x(A,(r))) 5 0

for all T > 0.

such that (5.7) and (5.14) hold.

n-m OIIJT

(c) For each T > 0, there exists {A,} c A' (possibly depending on T)

5.4 Remark In conditions (b) and (c) of Proposition 5.3, (5.14) can be replaced by

( 5 . 1 4 ) lim SUP dX,(A,(t)). 41)) = 0.

Denoting the resulting conditions by (b) and (c'), this is easily established by 0

n-rD O s r s T

checking that (b) is equivalent to (b) and (c) is equivalent to (c').

Proof. (a r* b) Assuming (a) holds, there exist {A"} c A and {u,} c (0,oo) such that (5.6) holds, u,-+ GO, and d(x,, x, A,, u,,)-+ 0; in particular,

(5.15) lim sup r(x,(r A u,), x(A,(r) A u,)) = 0.

Given T > 0, note that u, 2 TVA,(T) for all n sufliciently large, so (5.15) implies (5.14).

(b =* a)

(5.16) lim sup q(x,(t A u), x(A,(t) I\ u)) = 0

for every continuity point u of x by (5.13) with u, > A,(u)Vu for each n. Hence (a) holds.

n-m 1 2 0

Let (A,) c A satisfy the conditions of (b). Then

n-oo 1 2 0

(b c) Immediate.

120 CoMaCENcE OF nOIAUUTY hlEAsuRES

(C a b) Let N be a positive integer, and choose {A,"} c A' satisfying the conditions of (c) with T = N, such that for each n, #(r) - R:((N) + r - N for all t > N. Define TO" = 0 and, for k = 1,2,. . . ,

T: = inf t > I : r(x(t), X(TIN_ ,)) > - (5.17) I N ' I if rr-, < co, T: = co if r:-l = a. Observe that the sequence {T:} is (strictly) increasing (as long as its terms remain finite) by the right continuity of x and has no finite limit points since x has left limits. For each n, let uca = (A,">-'(rf) for k 5 0, 1,. . . , where (A,")-'(a) = a, and define p." E A by

N N - I N (5m18) C(,"(t) = r: + ( 1 - uk.awuk+I .a - $a) b k + 1 - 7:h t E Cut a , 4'+ 1. J CO, NI, k = 0, I , . . . ,

p:(t) = C(,N(N) + t - N , t > N,

where, by convention, 00 - I 00 = 1. With this convention,

(5.19) Y(P.") = max I log (4'+ 1, a - uL". A - '(4'+ 1 - 4')I NI.4.. < N

and

(5.20) SUP dXa(t), xb,N(t))) OsrsN

s SUP W r ) , x(C'(~))) + SUP 4x(C'(t)), X(p,N(t))) O s r s N O S I S N

2 5 SUP dxa(t), x(X'(t))) +

O s r s N

for all n. Since uca = T: for k = 0, I , . . . , (5.18) implies that lim,,,y(p,") = 0, which, together with (5.20) and (5.14) with T = N, implies that we can select 1 < n, < n, < - - - such that y(r(,", s 1/N and S U ~ ~ ~ , ~ ~ ~ X , , ( ~ ) , x(p,"(t))) 5 3 / N for all n 2 nN. For I s n < n , , let 1" E A be arbitrary. For nN S n < nN+l, where N 2 I, let If,, = p.". Then {If,,) c A satisfies the conditions of (b). 0

5.5 Corollary For x, y E Ds[O, a), define

(5.21) d'(x, y) = inf [q(x(r A u), y(A(c) A u)) AeA'

v ( I n ( t ) A u - t A u l A l ) ] d u .

The d' is a metric on DdO, a)) that is equivalent to d. (However, the metric space (DEIO, a), d') is not complete.)

5. THE SPACE D,lO, ad 121

Proof. The proof that d' is a metric is essentially the same as that for d. The equivalence of d and d' follows from the equivalence of (a) and (c) in Proposi- tion 5.3. We leave the verification of the parenthetical remark to the reader. 0

5.6 Theorem plete, then (DJO, 00). d) is complete.

If E is separable, then DEIO, 00) is separable. If (E. r ) is com-

Proof. Let {a,} be a countable dense subset of E and let functions of the form

be the collection of

(5.22)

where 0 = to < I , < + 3 c t, are rationals, i , , . . . , in are positive integers, and n 2 1. We leave to the reader the proof that r is dense in DJO, 00) (Problem 14).

To prove completeness, it is enough to show that every Cauchy sequence has a convergent subsequence. If {x,} c D,[O, 00) is Cauchy, then there exist I s N , < N, < . . . such that m, n 2 Nk implies

(5.23) d(x,, x,) s 2 - k - 1 e - k .

For k = I , 2,. , . , let Y k = X N , and, by (5.23)- Select Uk > k and Ak E A such that

(5.24) y(&)vd(yk, ykt19 u k ) 2 - ' ;

then, recalling (5.5).

(5.25)

exists uniformly on bounded intervals, is Lipschitz continuous, satisfies

(5.26)

and hence belongs to A. Since

(5.27) sup d Y k ( p i '(I) uk)* Y k t l(h-+!l(r) A uk) I 2 0

= dYk(p; '(l) A uk)* Y k + l(Ak(p; '(I)) A uk)) 120

= 4( Y k ( l uk), Y k + A ud) I 2 0

5 2 - k

for k = I , 2 , . . . by (5.24). it follows from the completeness of (E, r ) that zk ZE yk 0 Irk- ' converges uniformly on bounded intervals to a function y: [0, a)+ E.

122 CONVMGENCE Of MOMBlUTY MEASUUS

But each * i t ) = Oand

(5.28) lim SUP d M i '(Oh NO) = 0

for all T > 0, we conclude that limk-,md(y,, y ) = 0 by Proposition 5.3 (see Remark 5.4). 0

E Ds[O, a), so y must also belong to Ds[O, 00). Since Iim,--

it-m Osl5T

6. THE COMPACT StTS OF &lo, 00)

Again let (E, r) denote a metric space. In order to apply Prohorov's theorem to @(D,[O,oo)), we must have a characterization of the compact subsets of Ds[o, 00). With this in mind we first give conditions under which a collection of step functions is compact. Given a step function x E DEIO, a), define so(x) = 0 and, for k = 1,2,. . . , (6.1) sk(x) = inf { t > sk- I(x): x(t) # x(c-))

ifs,-,(x) < oo, SAX) = 00 ifs,-,(x) = 00.

6.1 lemma Let r c E be compact, let 6 > 0, and define A(r, 6) to be the set of step functions x e &lo, 00) such that x(t) e r for all c 2 0 and s&(x) - sk- i(x) > S for each k 2 I for which sk, ,(x) < m. Then the closure of

A(T, 8) is compact.

Proof. It is enough to show that every sequence in A(T, 6) has a convergent subsequence. Given {x,} c A(T, S), there exists by a diagonalization argument a subsequence { y,} of {x,,} such that, for k = 0, 1,. . . , either (a) sit(ym) < m for each m, lim,,,-m sk(y,,,) E t k exists (possibly a), and y,(sk(y,)) E a, exists, or (b) sit( y,) = oo for each M. Since sk( y,) - sk - I ( y,,,) > 6 for each & 2 I and m for which sk-,(y,,,) < 00, it follows easily that (y,) converges to the function y E DEIO, a) defined by f i t ) = a,, zk s t < t k + t , k = 4 1 , . . . . 0

The conditions for compactness are stated in terms of the following modulus of continuity. For x B Ds[O, a), 6 > 0, and T > 0, define

(Id i 1. I Ill- I . Ill (6.2)

where {t ,} ranges over all partitions of the form 0 = to c t i c - * * < r,- < T s I, with mini s,s,(r, - I,- > S and n 2 1. Note that w'(x, 6, T) is nonda creasing in 6 and in T, and that

w'(x, S, T) = inf max sup r(x(s), x(I)),

WYX, 6, T) s w'(Y, 6, T) + 2 SUP M s ) , As)). O s r c T + b

(6.3)

6. TM COWACTSETSOF DP, a01 123

6.2 Lemma (a) For each x B DJO, 00) and T > 0, w’(x, 8, T) is right continuous in 6 and

lirn w’(x, 8, T) = 0. &-O

(b) If {x , } c DECO, oo), x E DEIO, oo), and d(xm, x ) = 0, then

for every 6 > 0, T > 0, and E > 0. (c) For each 6 > 0 and T > 0, w’(x, 6, T) is Bore1 measurable in x.

Proof. (a) The right continuity follows from the fact that any partition that is admissible in the definition of w’(x, 6, T) is admissible for some 6’ > 6. To obtain (6.4), let N 2 1 and define {tr} as in (5.17). If 0 < 6 < min {tr+ - t,”:t: < T}, then w‘(x, S, T) 5 2 / N .

(b) Let {x,} c DJO, ao), x B DEIO, oo), 6 > 0, and T > 0. If 1imndQ d(x,, x ) = 0, then by Proposition 5.3, there exists {A,} c A’ such that (5.7) and (5.14) hold with T replaced by T + 6. For each n, let y,,(t) = x(A,(r)) for all t 2 0 and 6, = supOs,sr[A,(f + 6) - A,,(r)]. Then, using (6.3) and part (a),

(6.6) - - lirn w‘(x,, 6, T) = lirn w’(ym, 6, T) n-m 11-9)

5 lim w’(x, 8,. 1AT)) 1-4)

s lirn w’(x, 6, V 6, T + E )

= w‘(x, 6, T + E )

n- m

for all E > 0. (c) By part (b) and the monotonicity of w‘(x, 6, T) in T, w’(x, 8, T+)

ES lirn,,,, w’(x, S, T + 6) is upper semicontinuous, hence Bore1 measurable, in x. Therefore it suflices to observe that w‘(x, 6, T) = lirn,,,, w‘(x, 6,

0 ( T - E ) +) for every x c DEIO, 00).

6.3 Theorem Let (E, r) be complete. Then the closure of A c Ds[O, 00) is compact if and only if the following two conditions hold:

(a) For every rational t 2 0, there exists a compact set r, c E such that

(b) For each T > 0, ~ ( t ) E r, for all x E A.

lirn sup w’(x, 6, 7’) = 0. &-0 r e A

124 CONVERGENCE OF MOWUTY MUSURS

6.4 Remark In Theorem 6.3 it is actually necessary that for each T > 0 there exist a compact set rT c E such that x(t) E T r for 0 s t s T and all x E A. See Problem 16. 0

Proof. Suppose that A satisfies (a) and (b), and let 12 1. Choose 6, E (0, 1) such that

and m, 2 2 such that l/m, < 6,. Define Y') = U~L+~)'"rT,,ml and, using the notation of Lemma 6.1, let A, = A(T"), 6,).

Given x E A, there is a partition 0 - to < t l < - * < t,- < 1 s t , < 1 + I < In+ , = co with min,,,,,(t, - r , - l ) > 6, such that

(6.9)

Define x' E A, by x'(t) = x(([m, r ] -t- l)/m,) for t , s t < t , , i = 0, 1,. . . , n, Then suPo~:<lr(xl(l)r xo) s 2/r, so

(6.10) d(x', x) s e-" sup [r(x'(t A u), x( r A u)) A 13 du I20

s 211 -+ e-' < 311.

It follows that A c A:". Now I was arbitrary, so since A, is compact for each 1 2 1 by Lemma 6.1, and since A c nrr, A;',, A is totally bounded and hence has compact closure.

Conversely, suppose that A has compact closure. We leave the proof of (a) to the reader (Problem 16). To see that (b) holds, suppose there exist q > 0, T > 0, and {xn} c A such that w'(xe, l/n, T) 2 q for all n. Since A has compact closure, we may assume that lim,4ad(xn, x ) = 0 for some x E DEIO, a). But then Lemma 6.2(b) implies that

(6.1 1) q s lim w'(xn, S, T) s w'(x, 6, T + 1)

for all 6 > 0. Letting S-, 0, the right side of (6.1 I ) tends to zero by Lemma 0

n-(o

6.2(a), and this results in a contradiction.

We conclude this section with a further characterization of convergence of sequences in Os[O, m). (This result would have been included in Section 5 were it not for the fact that we need Lemma 6.2 in the proof.) We note that (C,[O, 00). d,) is a metric space, where

(6.12) ddx, y) = e -" sup Cr(x(t), y(9) A 1 I du. o s r s u

6. T M E C O k Y A C T ~ O F D,lO, m) 125

Moreover, if {x,} c CJO, oo) and x E C,[O, oo), then lim,-ta, dv(x,, x) = 0 if and only if whenever {t ,} c [0, a), t 2 0, and r(x,,(t,), x(t)) = 0. The following proposition gives an analogue of this result for

t, = t , we have

(4TCO,a) , 4.

6.5 Proposition Let (E, r) be arbitrary, and let (x,} c D,[O, 00) and x E D,[O, a). Then d(x,, x) = 0 if and only if whenever (1,) c 10, w), t 2 0, and t, = t , the following conditions hold:

(a)

(b) If s, = t , then

(c) If limm+a,r(x,(t,), x(f-)) = 0, 0 s s, 5 t , for each n, and

Nx,,(t,,h x(r)) A r(x,(r,,), x(r -)) = 0.

r(x,(r,,), x(t)) = 0, s, 2 t, for each n, and limn+,,, r(x,(s,), x(t)) = 0.

s, = c, then r(x,,(s,), x(t -)) = 0.

(6.15) 4x(UQ), ~ ( t ) ) S SUP 4x(Uu))* xn(u)) 0suS.r

+ 4 X n ( t n h x(l))

for each n. If also r(x(A,(r,,)), x(r)) = 0 by (6.15), so since A&) 2 A,,(t,) for each n and lim,+mA,,(s,) = r , i t follows that limndm r(x(A,(s,)), x(c)) = 0. Thus, (b) follows from (6.14). and the proof of (c) is similar.

We turn to the proof of the sufliciency of (aHc). Fix T > 0 and for each n define

r(x,(r,), x(t)) = 0, then

(6.16) E, = 2 inf { E > 0 : r(r, n, 8 ) z 0 for 0 5 r 5 T } ,

126 CONMCEHCE OF nOUIlUTY MLASUIES

where

(6.17) r(t, n, E ) = {S E ( t - E, t + E ) n [O, 00): ~(x,(s), dr)) < e,

r(X& - 1, x(l- 1) E } .

We claim that limn-,.,, E, = 0. Suppose not. Then there exist 6 > 0, a sequence {flk) of positive integers, and a sequence {tk} c [O, T) such that r(tk, f l k , E )

= 0 for all k. By choosing a subsequence if necessary, we can assume that limk+m f k = t exists and that t , < t for all k, f k > t for all k, or f k = t for all k. In the first case, limk,, x(tr) = limk-m x(tk-) = x( t - ) , and in the second case, lirnk-.,.,, x(fk) = limk-m %(tk -) = x(t). Since (a) implies that limn-,,, x,(s) = limn4mxn(s-) = x(s) for all continuity points s of x, there exist (by Lemma 5.1 and Proposition 5.2) sequences (a,} and (6,) of continuity points of x such that a, c t < b, for each n and a n d t and 6,- t suliiciently slowly that limn+w x,(a,) = limn-.,.,, x,,(a,-) = limn-- x(a,) = x ( t - ) and limn+- x,(b,) = x,(b,-) = limn*- x(b,) = x(t). If t k < t (respectively, f k > r ) for all k, then a, (respectively, b,,) belongs to r(rk, mk, E ) for all k suficiently large, a contradiction. It remains to consider the case f k = I for all k. If x(t) = x ( r -), then t 6 r(t, fib, E ) for all k sufficiently large by condition (a). Therefore we suppose that tfx(t), x( t - ) ) = 6 > 0. By the choice of {a,) and {b,} and by condition (a), there exists no 2 1 such that for all n 2 no,

(6.18)

(6.20)

By (6.18), a, c s, S b,, and therefore s,, E (t - E. r + e), r(xm(sn), x(t)) s (6 A ~)/2, and r(x,(s, -), x(t)) 2 (Sh ~)/2. The latter inequality, together with (6.19), implies that r(x,,(s,--), x ( t - ) ) < (6 A E ) / ~ . We conclude that s, E r(t, n, 8) for all n 2 no, and this contradiction establishes the claim that limn-.m E,, = 0.

1'; < * * * < r&- , < T s tk with min, s,sm,,(tr - t:- ,) > 36, such that

(6.21) max sup r(x(s), Nt)) 4 WIX, 3 ~ " , T) + E n ,

and put m.' = max { i 2 0 : t; s T} (m: is m, - 1 or m,). Define 1,(0) = 0 and A&;) = inf J-(r:, n, E,) for f = 1,. . . , m:, interpolate linearly on [O, f"n:J, and let A&) = t - t:: + A&::) for all t > r::. Then A, E A' and sup, L o Ilz,(r) - r l s E,, .

We claim that limn-- supos,,,r(x,,(1,(t)), x(r)) = 0 and hence lima-m d(x,, x) = 0 by Proposition 5.3 (see Remark 5.4). To verify the claim, it is enough

For each n, we construct 1, E A' as follows. Choose a partition 0 = r: <

I S l + n . l.Ictr:.,.m

7. CONVEIIGENCE IN DISTUBUIION IN DJO, Q) 127

to show that if It,} c [O, TI, 0 s t s T, and limn4mrn = I, then lim,4m r(x,,(A,(c,)), x(t,)) = 0. If x ( r ) = X(C -), the result follows from condition (a) since limm-m A,&,,) = t . Therefore, let us suppose that X(t) # x(c-). Then, for each n sufliciently large, r = rym for some in ts { 1,. . . , m:} by (6.21) and Lemma 6.2(a). To complete the proof, it sufices to consider two cases, {I ,} c [ r , T ] and {t,} [O, r). In the first case, A,(r,) 2 A&) - A,,(r;.) and dx,(A,,(r;)), x(t)) s E, for each n suficiently large, so r(x,(A,(t,)), x(c)) = 0 by condition (b), and the desired result follows. In the second case, A&,) < A&) = A$;,) and either r(xn(A,,(~ym)-), x(t -)) < E, or r(xm(An(r;)), x(r -)) s c,, (depending on whether the infimum in the definition of An(r:a) is attained or not) for each n sufficiently large. Consequently, for such n, there exists s, with A,(t,) < s, s A&;) such that r(x,(s,), x(t -)) s en, and therefore limn-.- ~xm(A,,(r,)), dt-)) = 0 by condition (c), from which the desired result follows. This completes the proof. 0

7. CONVERGENCE IN DISTRIBUTION IN DB[O,aJ)

As in the previous two sections, ($ r) denotes a metric space. Let 9, denote the Borel o-algebra of Dt[O, 00). We arc interested in weak convergence of elements of P(DEIO,a)) and naturally it is important to know more about Y E . The following result states that 9, is just the a-algebra generated by the coordinate random variables.

7.1 Proposition For each t 2 0, define n, : D,[O, 00)- E by n,(x) = x(t). Then

(7.1)

where D is any dense subset of [O, a). If E is separable, then .4ps = 9’;.

Y E 3 9; 5 dn,: 0 s r < 00) = dn,: t E D),

Proof. For each E > 0, t 2 0, and/€ c((E),

(7.2)

defines a continuous functionfi on DJO. 00). Since Iirn,+ J x x ) =/ (R, (x ) ) for every x E DEIO, a), we find that f o n, is Borel measurable for every J E C(E) and hence for everyJE B(E). Consequently,

(7.3) n; ‘(r) = {x E DsEo, a): xr(n,(x)) = I 1 E Y&

for all r E WE), and hence Y E = YE. For each t 2 0, there exists {c,,} c D n I t , ao) such that limn+m t , = t . Therefore, A, = limn-m n,. is dn,: s E D)- measurable, and hence we have (7.1).

Assume now that E is separable. Let n 2 1, let 0 = to < t , < . - < t, <

i = O , l , ..., n.

la+ = ao, and for a. , a,, . . . , a,, E E define Hao, a I , . . . , a,,) E D,[O, a) by

(7.4)

Since

q(ao,al , ..., a,)(t) = al , r, s t < t i + ] ,

d(rt(ao , a ] , . . . , a,), , a’,, . . . , a:)) s max dotl, a;), Osisn

(7.5)

tf is a continuous function from En+’ into DEIO, a). Since each n, is Y’,-measurable and E is separable, we have that for fixed z E DEIO, a), d(z, q 0 (n,,, nt,, .. ., n,J is an Ycmeasurable function from DJO, a) into R. Finally, for m = I, 2,. . . , let )I,,, be defined as was q with t l = i/m, i = 0, 1, ..., n f m2. Then

lim 4 2 , qm(qo(x), . . . , nlm2(x))) = d(z, x) 111-m

(7.6)

for every x E D,[O, 00) (see Problem 12). so d(z, x) is Y6-measurable in x for fixed z E DEIO, a). In particular, every open ball B(z, e) = { x E DJO, 00): d(z, x) < E} belongs to YE, so since E (and, by Theorem 5.6, Dr[O, a)) is separable, 9; contains all open sets in D,[O, ao) and hence contains 9,. 0

A DEIO, a)-valued random variable is a stochastic process with sample paths in DEIO, co), although the converse need not be true if E is not separable. Let { X J (where a ranges over some index set) be a family of stochastic processes with sample paths in D,[O, 00) (if E is not separable, assume the X, are DEIO, a)-valued random variables), and let { P a } c 9 ( D E [ 0 , 00)) be the family of associated probability distributions (i.e., P,(B) = P { X , E B } for all B E LYE). We say that {X, } is relutiuely compact if { P a ) is (i.e., if the closure of { P a } in a D 6 [ 0 , 00)) is compact). Theorem 6.3 gives, through an application of Prohorov’s theorem, criteria for {X,,} to be relatively compact.

7.2 Theorem Let (E, r) be complete and separable, and let { X a } be a family of processes with sample paths in D,[O, ao). Then {X, } is relatively compact if and only if the following two conditions hold:

(a) For every q > 0 and rational t 2 0, there exists a compact. set rqe, c E such that

(7.7)

(b) For every q > 0 and T > 0, there exists 6 > 0 such that

sup P{W’(X,, 6, T) 2 tf} 5 ‘I. a

7. CONVERGENCE IN D(STIIIIUTI0N IN DJO, o) 129

7.3 Remark In fact, if ( X , } is relatively compact, then the stronger compact containment condition holds; that is, for every q > 0 and T > 0 there is a compact set I-qs r c E such that

(7.9) inf P(X,(t) E rq. for 0 s t I T } 2 1 - q. 0: 0

Proof. If { X , } is relatively compact, then Theorems 5.6, 2.2, and 6.3 immediately yield (a) and (b); in fact, I-:, , can be replaced by I-,,, in (7.7).

Conversely, let E > 0, let T be a positive integer such that e - r < c/2, and choose 6 > 0 such that (7.8) holds with q = ~ / 4 . Let m > 1/6, put r = u;IT,, re2-l-2,1/m, and observe that

(7.10)

Using the notation of Lemma 6.1, let A = A(T‘, 6). By the lemma, A has compact closure.

Given x E D,[O, 00) with w’(x, 6, T) < 44 and x(i/m) E for i = 0, I , ..., mT, choose 0 = lo < t , < . . * < t , - , < T s I, such that min, si& - t i - ,) > 6 and

& inf P{X, ( i /m) E re? i = 0, I , . .., m r } 2 I - -. a 2

E (7.1 I ) max sup W s ) . ~ ( 0 ) < 4.

and select { y l ) c r such that r(x(i/m), y,) < 4 4 for i = 0, 1, . . . , mT. Defining x‘ E A by

I s i s n 3. f 6 I f i - I. I t )

we have ~ u p ~ ~ ~ ~ ~ T ( x ( t ) , x’(t)) < ~ / 2 and hence d(x, x’) < 4 2 -t e-’ < E, implying that x E A‘. Consequently, inf,P(X, E A‘} 2 I - E , so the relative com-

0 pactness of {X,) follows from Theorems 5.6 and 2.2.

7.4 Corollary Let (E, r ) be complete and separable, and let {X, , } be a sequence of processes with sample paths in D,[O, 00). Then ( X , , } is relatively compact if and only if the following two conditions hold:

(a) For every q > 0 and rational t 2 0, there exists a compact set rq,, c E such that

(7.13)

(b)

- lim P(X,(t) E. r;,,} 2 1 - q . n-m

For every q > 0 and T > 0, there exists 6 > 0 such that

(7.14)

Proof. Fix q > 0, rational t 2 0, and T > 0. For each n 2 1, there exist by Lemmas 2.1 and 6.2(a) a compact set r, c E and 6, > 0 such that P{X,,(r) E r:) 2 I - q and P { w'(X, , 8, , T) 2 q } s q. By (7.1 3) and (7.14), there exist a compact set ro c E, So > 0, and a positive integer o0 such that

(7.15)

and

(7.16)

inf P{X,(t) E rg} 2 1 - q a z n o

We can replace no in (7.15) and (7.16) by 1 if we replace To by r = u7;o' r, 0 and So by S = A:";' S,,, so the result follows from Theorem 7.2.

7.5 Lemma Let (E, r ) be arbitrary, let TI c r, c sequence of compact subsets of E, and define

- be a nondecreasing

(7.1 7)

Let {X,} be a family of processes with sample paths in S. Then {X,} is relatively compact if condition (b) of Theorem 7.2 holds.

S = {x E DEIO, a): x(t) E r,, for 0 s r s n, n = 1, 2,. . .}.

Proof. The proof is similar to that of Theorem 7.2 Let E > 0, let T be a positive integer such that e-' < 42, choose 6 > 0 such that (7.8) holds with q = ~ / 2 , and let A = A ( r T , 6). Given x E S with w'(x, 6, 7') < e/2, it is easy to construct x' e A n S with d(x, x') < E, and hence x E (A n Sr. Consequently, inf,P(X, E ( A A Sy) ;r I - E, so the relative compactness of {X,} follows from Lemma 6.1 and Theorem 2.2. Here we are using the fact that (S, d) is complete and separable (Problem 15). 0

7.6 Theorem Let (E, r) be arbitrary, and let {X,} be a family of processes with sample paths in DJO, 00). If the compact containment condition (Remark 7.3) and condition (b) of Theorem 7.2 hold, then the X, have modifications 2. that are DEIO, 00)-valued random variables and (2,) is relatively compact.

Proof. By the compact containment condition there exist compact sets r, c E, n = 1, 2, ..., such that inf, P{X,(r) E r, for 0 s t s n } 2 1 - n - ' . Let E , = u,, rn. Note that E, is separable and P{X,(r) E E,,} = I so X, has a modification with sample paths in DEo[O, a). Consequently, we may as well assume E is separable. Given 4 > 0, we can assume without loss of generality that

(7.18) S, = (x E D,[O, 00): x(c) E

-". ,,} is a nondecreasing sequence of compact subsets of E. Define

for 0 s r s n, n = 1, 2,. . .}.

7. CONVUCENQ IN CMSfR((IWION IN DJO, OD) 131

and note that inf, P { X, E S,} 2 1 - q. By Lemma 7.4, the family { P:) c 9(S,), defined by

(7.19) P p ) = P{X, E B J X , E S,},

is relatively compact. The proof proceeds analogously to that of Corollary 2.3. We leave the details to the reader. 0

7.7 Lemma If X is a process with sample paths in Ds[O, a), then the complement in 10, a ~ ) of

(7.20)

is at most countable.

D ( X ) s {r 2 0: P { X ( t ) = X(c-)} = I }

Proof. Let E > 0.6 > 0, and T > 0. If the set

(7.21)

contains a sequence {r,} of distinct points, then

(7.22)

contradicting the fact that, for each x E DBIO, a), r(x(t), x(t -)) 2 E for at most finitely many t E [O, TI. Hence the set in (7.21) is finite, and therefore

(7.23)

is at most countable. The conclusion follows by letting E-+ 0.

(0 s r 5 T : P(r(X(t), X(r-1) 2 E } 2 a)

P{r(X(t,), X(r,-)) L E infinitely often} 2 6 > 0,

{ t 2 0: P{r(X(r), X(r-)) 2 E } > O}

0

7.8 Theorem Let E be separable and let X,, n = I , 2, . . . , and X be processes with sample paths in D,[O, m).

(11 If X, * X , then

(7.24) (XAt i 1, * * * * x At J ) z+ (x(t i 1, * * * 9 X(rk)) for every finite set { t I . ..., t k } c D(X). Moreover, for each finite set { t i , ..., t,) t [0, 00). there exist sequences ( I ; } c It,, a), ..., {r;} c [ r k , 00) converging to t , , . . . , r k , respectively, such that (X,(t'i), . . . , X,(t:)) 4 (X(t , ) , . . . , X(tk)).

(b) If {X,} is relatively compact and there exists a dense set D c [O, 00)

such that (7.24) holds for every finite set ( i l l . . .. r k } c D, then X, rg X .

Proof. (a) Suppose that X , rg X. By Theorem 1.8, there exists a probability space on which are defined processes V , , )t = 1, 2, . . . , and Y with sample paths in Ds[O, 00) and with the same distributions as X, , n = 1, 2, ..., and X , such that lim,,-md(x, Y) = 0 as. If t E D(X) = D(Y), then, using the

132 CONVERGENCE OF ROMBIUTY MUSWES

notation of Proposition 7.1, n, is continuous as. with respect to the distribution of Y, so Y,(f) = Y(t) 8.5. by Corollary 1.9, and the first conclusion follows. We leave it to the reader to show that the second conclusion is a consequence of the first, together with Lemma 7.7.

(b) It suffices to show that every convergent (in distribution) subsequence of {X,} converges in distribution to A'. Relabeling if necessary, suppose that X, * Y. We must show that X and Y have the same distribution. Let {r,, . . . , t k } c D(Y) and f,, . . . , 1; E C(E), and choose sequences { t ;} c D n [tl, a), . . , {rr} c D n [ t k , 00) converging to r i , . . . , f k , respectively, and n, < n, < n3 < * * such that

Then

for each m 2 1. All three terms on the right tend to zero as m+ 00, the first by the right continuity of X, the second by (7.25), and the third by the facts that X,, - Y and { t i , . . . , r k } c my). Consequently,

(7.27)

for all { t , , ..., t r ) c [O, 00) (by Lemma 7.7 and right continuity) and all f,, . .., 1; E c (E) . By Proposition 7.1 and the Dynkin class theorem (Appendix 4). we conclude that X and Y have the same distribution. 0

8. CRlTERtA FOR REUTIVE COMPACTNESS IN Dal0,00)

Let (E, r) denote a metric space and q = r h 1. We now consider a systematic way of selecting the partition in the definition of w'(x, 6, T). Given E > 0 and x E DJO, a), define r0 = uo = 0 and, for k = 1,2,. . . ,

8. CRITERIA FOR R E U T M COMIACWESS IN D,,IO, a01 133

if Ty, - 1 < m, 'Ck = if Tk - 1 = 00,

n k = sup t 5 T k : r(x( l ) , x ( T k ) ) v r(x(f - ) v x ( ? k ) ) 2 - (8.2) I 2 7 if ?k < 00, and oh = 00 if Tk 3: 00. Given 6 > 0 and T > 0. observe that w'(x, 6, T ) < c/2 implies min { ? k I - o k : ?k < T } > 6, for if T& I - uk 5 6 and t k c T for some k 2 0, then any interval [a, 6) containing ?k with 6 - a > 6 must also contain o k or T k + , (or both) in its interior and hence must satisfy sup., ,* Is, 6, r(x(s), x( t ) ) 2 ~ / 2 ; in this case, w'(x, 6, T) 2 4 2 .

Letting

for k = 0, 1, . . . , we have limh-,a Sk = 00. Observe that, for each k 2 0,

(8.4) ak < sk T k 5 n k t I s k + I s T k + I *

and

if s k < 00, where the middle inequality in (8.4) follows from the fact that r ( X ( T k ) , x ( ? k + I ) ) 2 ~ / 2 if ? k + I c 00. We conclude from (8.5) that min { T k

?k < T + 6/2} > 6 implies -

for if not, there would exist k 2 0 with s k c T, t k 2 T + 612, and s, + I - sk s 6/2, a contradiction by (8.4). Finally, (8.6) implies w'(x, 6/2, T) 5 E.

Let us now regard t k , O k , and s k , k = 0, 1,. . . , as functions from DJO, Q)) into [0,003. Assuming that E is separable (recall Remark 3.4). their 9,-measurability follows easily from the identities

(8.7)

{ ? k < u} = ( T k - 1 < a} n u t c (0 , u ) n Q

and

(8.8) { n k 2 u} = ( T k = m) U l ' ( X ( U - ) , X ( ? k ) ) 2

134 CONVERGENCE OF mLouuuTv ~ ~ ~ w l l l ~ o

valid for 0 

Let (E, r) be separable, and let ( X m ) be a family of processes paths in D,[O, a). Let zFa, u : ~ , and .$ma, k = 0, 1,. . . , be defined

t 0 and X , as in (8.1H8.3). Then the following are equivalent:

lim inf P{w’(Xm, 6, T) < a } = 1, (8.9)

(8.10) lim inf P(min {s$,!, - s;*’: s:‘ < T} L S} = 1,

(8.11) lim inf P{min {I?,!, - a;*#: T i . ‘ < T} 2 S} = I ,

E > 0, T > 0.

E > 0, T > 0.

E > 0, T > 0.

( - 0 m

6-0 a

d-.O m

Proof. See the discussion above. 0

8.2. Lemma For each a, let 0 = s: < fl < 6 < * be a sequence of random variables with limb-,,,, s: = 00, define A: = g+ , - si for k = 0, I,. . . , let T > 0, and put K,( T) = max (k 2 0: st < T}. Define F: [O, 00) --+ [O, 11 byF(t) = s~pmsup~,o P{A; < I, < 7’). Then

(8.12) F(6) 5 sup P min A,O < 6 s LF(6) + e’ Le-”F(r) dt I l for all S > 0 and t = 1,2,. . . . Consequently,

(8.13) lim sup min A: < a} = 0

if and only if F(O+) = 0.

4-0 Q OShSiKdT)

Proof. The first inequality in (8.12) is immediate. As for the second,

min A: < 6 s P{A; < 6, s; < T} + P{K,(T) 2 L} (8.14) P( L - 1 1 k - 0 0 $ k s &(‘I

8. CRITERIA FOR ReAnn COMPACTNESS IN DAO, CO) 135

Finally, observe that F(O+) = 0 implies that the right side of (8.12) approaches zero as 6 4 0 and then L --+ 00. 0

8.3 Proposition Under the assumptions of Lemma 8.1, (8.9) is equivalent to

(8.15) lim sup sup P{Z:;’~ - < 6, r;.’ < T } = 0, E > 0, T > 0. d 4 0 d L Z O

proof. The result follows from Lemmas 8.1 and 8.2 together with the inequalities

(8.16)

The following lemma gives us a means of estimating the probabilities in (8.15). Let S(T) denote the collection of all (S;+}-stopping times bounded by T.

8.4 Lemma DJO, a), and fix T > 0 and p > 0. Then, for each 6 > 0.1 > 0, and T E S(T),

Let (E, r ) be separable, let X be a process with sample paths in

1% CONVERGENCE tx riOlUUUtY MEASU~ES

8.5 Remark 8 In (8.19), SUP,,^(^+^^) can be replaced by S U ~ ~ . ~ ~ ( ~ + ~ ~ , , where So(T + 26) is the collection of all discrete {9f}-stopping times bounded by T + 26. This follows from the fact that for each r e S(T + 26) there exists a sequence (7,) c So(T + 26) such that T" 2 T for each n and lim,,-,m r, = T ; we also need the observation that for fixed x E DJO, a), SUP0sc,r3dA, qB(x(t), x(t - u)) is right continuous in t (0 5 t < a).

(b) If we take 1 = e/2 E (0,l J and T = f k A T (recall (8.1)) in Lemma 8.4, where k 2 1, then the left side of(8.17) bounds (8.20) P { T k + 1 .- rk 6, f k - bk < 6, Tk < T } ,

which for each k 2 1 bounds P{?k+l - < 6, Tk < T, ?I > 6) . The left side of (8.18) bounds P { r , ~r; b}, and hence the sum of the right sides of (8.17)

0 and (8.18) bounds P ( T k + 1 - Uk < 8, Tk < T} for each k 2 0.

Proof. Given a (9:+}-stopping time 7, let M@) be the collection of .F:+-rneasurable random variables U satisfying 0 5 U 5 6. We claim that

(8.21) sup E[ sup q"(X(r + 01, X(T)V(X(r), X(r - u))] sup tSS(T+& U8Yd4) O S U S l d n r

~r; (a,, + 4a$C@).

To see this, observe that for each 7 E S(T + 6) and U E M#),

(8.22) 4 W ( T + U), X(d1 26

I a, ,6- '1 [q'(X(r + 8). X(r) ) + q8(X(7 + e), X ( T + V))] de

s a,, 8 - I[ iz* q ~ x ( 7 + e), x(t)) de

qp(x(r + u + el, x(f + 0)) de , 1 and hence

also, r + U E S(T + 24, so (8.21) follows from (8.23). Given 0 < q 0: q ( X ( r + I), X ( T ) ) > I - q ) ,

(8.25) q”(X(7 + A A S), X(t ) )#(X(r) , X ( t - u))

5 a S q O ( X ( r + 6). X ( r ) h @ ( X ( t ) , X(r - 0))

+ ajq’(X(r + S), X ( r + AAS))qc(X(r + AAS) , X ( r ) )

+ a: qb(X(r + 6). X ( t + A A S))q’(X(r + A A 6). X ( r - u))

for 0 s u 5 S A t . Since t + A A S E S(T + S), S - A A S E Mr+bhd(S), and A A d + u s 26, (8.21) and (8.25) imply that

(8.26) €[ sup q b ( X ( t -k AAd), X(t))q”(X(r), X ( r - u)) 1 O s u S d A r

s [as + 2a3ap + 4a$]~(6).

But the left side of (8.17) is bounded by (A - q)-@A-@ times the left side of (8.26), so (8.17) follows by letting q -+ 0.

(8.27) q Z c ( X ( A A 4, X ( 0 ) ) 5 c ~ ~ [ q ” ( x ( S ) , X ( A A S))qc(X(A A 6). X(0))

Now define A as in (8.24) with t = 0. Then

+ q”(X(6). X(0))q’(X(A A 6). X(0)) l

5 a8 Sp(x(6), x ( A A S))qp(X(A A 6). x(0)) + a# qO(X(d) , x(o)), so (8.18) follows as above. 0

8.6 Theorem Let (€, r ) be complete and separable, and let ( X , } be a family of processes with sample paths in Ds[O, 00). Suppose that condition (a) of Theorem 7.2 holds. Then the following are equivalent :

(a) { X J is relatively compact.

138 CONVEICENCEOFMOW~UTV~YAU~~DS

(b) For each T > 0, there exist /? > 0 and a family {ye@): 0 c 6 < I, all a} of nonnegative random variables satisfying

(8.28) ECqp(Xe(t + u), XAt)) I K k ’ ( X d f ) , XAf - 0)) ECYe(4 193 for 0 s t 5; T,O s u s 6, and 0 s u 5 6 A t , where 9: = 9 P ; i n addition,

(8.29) lim sup ECyAS)] = 0 d-0

and

(8.30)

(c) For each T > 0, there exists # > 0 such that the quantities

Jim sup E[q@((X,(6), X,(O))] = 0. 1-0 u

(8.31) C,(6) =

SUP SUP .[ SUP qC(Xg(r 4- U h Xa(t )hp(X, (d , X,(r - d)] , rrS37 l Osrsd O s v s i A r

defined for 0 < 6 < 1 and all a, satisfy

(8.32) lim sup Cg(6) = 0; &-0

in addition (8.30) holds. (Here S,(T) is the collection of all discrete (*:}-stopping times bounded by T.)

8.7 Remark (a) If, as will typically be the case,

(8.33) ECqp(Xe(t + u), XAtN I SfJ s ECYJ~) I $:I in place of (8.28), then E[:q0(XJ6), XAO))] s E[y#)] and we need only verify (8.29) in condition (b).

(b) - For sequences {X,,}, one can replace sup, in (8.29), (8.30), and (8.32) by as was done in Corollary 7.4. 0

Proof. (a ;r) b) In view of Theorem 7.2, this follows from the facts that

(8.34) d X g ( t + u), Xe(t)k(XAt) , XAt - u))

S dxa0 + 4, xAtN A d X A f ) , XAf - 0))

s w’(Xe, 26, T + 6)A 1

for 0 s t 5 T,O s u $ &and 0 5 u s dht, and (8.35) 4(XA4, XU(0)) s W W . , 6. T ) A 1.

(b ‘+ c) Observe that f in (8.28) may be replaced by t E S,(T) (Problem 25 of Chapter 2), and that we may replace the right side of (8.28) by its

a CMRU FOI REUM CWACTNESS IN 0110, 1s

supremum over u E [O, 6 A I] n Q and hence over u E [O, b A TI. Conse- quently, (8.29) implies (8.32).

(c 3 a) This follows from Lemma 8.4, Remark 8.5, Proposition 8.3, and Theorem 7.2. 0

The following result gives suficient conditions for the existence of {ya(S): 0 < S < I, all a} with the properties required by condition (b) of Theorem 8.6.

8.8 Theorem Let (E, r) be separable, and let {X, } be a family of processes with sample paths in Dc[O,ao). Fix T > 0 and suppose there exist /l> 0, C z 0, and 8 > I such that for all a

(8.36) ECqP(X.(t + h), Xe(t)) A q6(Xa(t), x e ( t - h))I s the,

o s t s r + 1, o I; h 5 t,

which is implied by

(8.37) E[qC’2(Xa(t + h), X&))qb’z(Xa(r), X,(t - h))] s Che,

Osrs;T+ 1,OI;hsr.

Then there exists a family {ya(b): 0 < b < I, all a} of nonnegative random variables for which (8.29) holds, and

(8.38) qYXa(r + u), Xa(t )MXa( t ) , Xa(t - v)) s VJS)

for 0 5 t s T, 0 5 u s 6, and 0 s v s 6 Ar.

8.9 Remark (a) The inequality (8.28) follows by taking conditional expacta-

(b) Let E > 0, C > 0.8 > 1, and 0 < h s r, and suppose that

(8.39)

for all A > 0. Then, letting /l = 1 + e,

(8.40) ECqc(Xa(t + h), XeWA qP(Xa(t), X,(t - h))]

tions on both sides of (8.38).

P{r(Xe(t + h), X.(r)) 2 I , r(Xa(r), Xa(r - h)) 2 A } s A-’Che

= P{ qp(Xa(r + hh Xa(t)) L X , q’(X.(t), Xa(t - h)) Z X} dx

0

Pmof. We prove the theorem in the case f l > 1; the proof for 0 < f l s 1 is similar and in fact simpler since qP is a metric (satisfies the triangle inequality) in this case. In the calculations that follow we drop the subscript a. Define

140 CONVUCXNCE OF WOBAIlLITY MEASURES

(8.41)

9" = C q'(X((k + 1)2-"), X(k2-"))Aq@(X(kZ-"), X((k - 1)2-")) I srszqr+ 1)- 1

for m = 0, I , . . . , and fix a nonnegative integer n. We claim that for integers m 2 n and j , k , , k , , and k3 satisfying

(8.42) 0 S j Z - " 5 k,2-" < kz2-" K k,2-" S (j + 2)2-" s T + 1,

we have

(8.43)

(If 0 < /3 I; 1, replace q by qc and q,!I8 by q, in (8.43).)

implies that k , = j , k , = j + 1, and k , = j + 2, and

(8.44) q(X((j + 2)2-3, X ( 0 + 1)2'3)Aq(X((j + 1)2-"), X02-" ) ) I; q,"?

Suppose (8.43) holds for some m 2 n and 0 I; j 2 - " s k12-'"- I < k , 2-'"-l < k32- '"- l s (j + 2)2-" 5 T + 1. For i = 1, 2, 3, let e, = q(X(k;2-"), X(k,2-"- ' ) ) , where if k, is even, k; = kJ2, and if k, is odd, k; = (k, f 1)/2 as determined by (8.45)

Note that ei = 0 if k, is even and e, 5 q,!$ , otherwise, so the triangle inequality implies that

(8.46)

q(X(k32-"), X(k,2-"))Aq(X(kz2-"), X(k,2-'")) s 2 f t#@. i=r

We prove the claim by induction. For m = n, (8.43) is immediate since (8.42)

6, = q(X((ki + 1)2-"-'), X(ki2-"-'))Aq(X(k,2-"-'), X((k, - 1)2-"-')).

q(X( k3 2 -" - I), X(k2 2 -" - I)) A q(X(kz 2 - " - I), X(kl 2 - - I))

5 re3 + q(X(k3 2 -"), X(k; 2 -9 + 811

A [ E z + q(X(k; 2-"), X(k; 2 - 9 ) + 61 J 5; 2q;'t + q(X(k, 2 - "), X(k; 2 - ")) A q(X(k; 2 - "), X(k; 2 - ")).

By the definition of k; , we still have 0 1;j2-" I; k12-" I; k; 2-" I; k3 2-" s (j + 2)2-" S T + 1, and hence the induction step is verified.

and t I , t , , and t, are dyadic rational with t , - c, s 2-" for some n 2 1, then there exist j , m, k , , k z , and k , satisfying (8.42) and t, = k,2-". Consequently,

If 0 s t , < t2 < t , s T +

i = n

By right continuity, (8.47) holds for all 0 I; 1 , < c3 < t3 < T + $ with t , - t 5 2-". If 6 2 4, let y(6) = 1 ; if 0 < 6 < $, let nl be the largest integer n

9. FURTHER CRITERIA FOR RELATIVE CWIAClNESS IN D,lO, ao) 141

satisfying 26 < 2 -", and define y(6) = qnr. Since ab s a A b for all Q, b E: [O, I ] , we conclude that (8.38) holds. Also,

(8.48) E[y(G)] = 2 f ECq:'@l s 2 f E[I,J'/P i==W 4-nr

m

5 2 c [2'(T + 1)C2-"]"b. l e n d

so lim,,-,, E[y(6 ) ] 3: 0 (and the limit is uniform in a). 0

8.10 Corollary Let (E, r) be complete and separable, and let X be a process with values in E that is right continuous in probability. Suppose that for each T > 0, there exist /I > 0, C > 0, and 0 > I such that

(8.49) E[qP(X(t +- h,), X ( t ) ) W ( x ( r ) . X ( t - h d 1 5 C(h, Vh,)'

whenever 0 S t - h, I; t s t +. h, s T. Then X has a version with sample paths in DEIO, 00).

Proof. Define the sequence of processes {X,} with sample paths in DEIO, a)) by X,( t ) = X ( ( [ n t ] + l ) /n). It suffices to show that {X,) is relatively compact, for by Theorem 7.8 and the assumed right continuity in probability of X, the limit in distribution of any convergent subsequence of { X , } has the same finite-dimensional distributions as X .

Given q > 0 and t 2 0, choose by Lemma 2.1 a compact set rqsl c E such that P { X ( t ) e r;.l} 2 1 - q. Then (7.13) holds by Theorem 3.1 and the fact that X,(t) * X(t) in E. Consequently, it suffices to verify condition (b) of Theorem 8.6, and for this we apply Theorem 8.8. By (8.49) with T replaced by T + 2, there exist p > 0, C > 0, and 8 > I such that for each )I

(8.50) EC&Xn(t + h), XAr)) A &Xn(l), Xn(l - h))I

[n t ] - [n(t - h) ] " [n(t + h)] - [nt]

.C( n n >. 0 5 l $ T + I , O S h h t .

But the left side of (8.50) is zero if 2h 5 I/n and is bounded by C(h + 5 3'Ch' if 2h > l/n. Thus, Theorem 8.8 implies that (8.29) holds,

0 and the verification of (8.30) is immediate.

9. FURTHER CRITERIA FOR RELATIVE COMPACTNESS IN DrlO, ao)

We now consider criteria for relative compactness that are particularly useful in approximating by Markov processes. These criteria are based on the following simple result. As usual, (E, r) denotes a metric space.

142 CONVERGENCE OF moornun MEAHI~ES

9.1 Theorem Let (E, r ) be complete and separable, and let (X,} be a family of processes with sample paths in D,[O, 00). Suppose that the compact containment condition holds. That is, for every 1 > 0 and T > 0 there exists a compact set rq, r c E for which

Let H be a dense subset of C(€) in the topology of uniform convergence on compact sets. Then {X,) is relatively compact if and only if { / o X a } is relatively compact (as a family of processes with sample paths in DJO, 00)) for each f E H.

Proof. GivenfE c(E) , the mapping x - j o x from D,JO, a)) into DJO, a) is continuous (Problem 13). Consequently, convergence in distribution of a sequence of processes {X,,) with sample paths in Ds[O, a) implies convergence in distribution of c / o A',,}, and hence relative compactness of {Xu} implies relative compactness of {fo X,}.

Conversely, suppose that { f 0 X,} is relatively compact for every f E H. It then follows from (9.1), (6.3), and Theorem 7.2 that { / o A',} is relatively compact for every f E i f (E) and in particular that {q(*,z) 0 X,} is relatively compact for each z E: E, where q = r A 1. Let 1 > 0 and T > 0. By the compactness of rq, r , there exists for each E > 0 a finite set { z ~ , . . . , z N ) c Tr, r such that min, s I s N 4 ( ~ , z,) < E for all x E rq,r. If y E rq, r , then, for some i E {I, . . . , N } , q(y, 2,) < E and hence

for all x E E. Consequently, for 0 5 t s T, 0 s u s 6, and 0 $ v s 6Ar .

where 0 < 6 < 1. Note that N depends on 1, T, and E.

9. FURW CRITERIA FOR RELATIVE COMIACTNESS IN DJO, 00) 143

Since lim,,osup,E[w'(q(*, z) 0 X , , 26, T + 6)h I] = 0 for each z E E by Theorem 7.2, we may select 9 and E depending on d in such a way that limd+o sup, E[y,(6)] = 0. Finally, (9.2) implies that

(9.4) ~ ~ ( 6 ) . X,(O)) 5 V IdX,(d), 2,) - M m ( 0 ) , zr)I + 2~ + ~w,(o)er, ,r l

for all S > 0, so limd,o sup, E[q(X,(S), X,(O))] = 0 by Theorem 8.6. Thus, the relative compactness of { X,) follows from Theorem 8.6. 0

N

i= I

9.2 Corollary Lct (E, r ) be complete and separable, and let ( X , } be a sequence of processes with sample paths in DJO, 00). Let M c c ( E ) strongly separate points. Suppose there exists a process X with sample paths in Dc[O, 00) such that for each finite set {g,, . . . , gk} c M,

(9.5) (g1, . . . , gk) X , e ( g 1 , . . . , g k ) 0 X in Dado, a>).

Then X , =. X .

Proof. Let H be the smallest algebra containing M u ( I } , that is, the algebra of functions of the form c!I la,/llJ12.. .A,,,, where 12 1, m z 1, and a, E R and /;, E M u ( 1 ) for i = 1 ,..., I and j = 1 ,..., m. By the Stone-Weierstrass theorem, H is dense in C ( E ) in the topology of uniform convergence on compact sets. Note that (9.5) holds for every finite set (gl,. . . , #k} c H. Thus, by Theorem 9.1, to prove the relative compactness of (X,} we need only verify (9.1).

Let r c E be compact, and let S > 0. For each x E E, choose {hf ,..., hi(,,} c H satisfying

E(x) = inf max (h:(y) - h f ( x ) l > 0,

and let U, = { y E E: mar, d l s k ( x ) Jh,'(y) - h,'(x)l < e(x)}. Then r c U x . r U x c r', so, since r is compact, there exist x , , . . . , x, E r such that r c Ur-, U,, c r? Define o: Dl[O, 00)- DaCO, 00) by a(xKt) = suporrS,X(~) and observe that u is continuous (by Proposition 5.3). For each n, let

(9.7)

y : r k y ) z d I S l S k ( r ) (9.6)

Yn(t) = min (gXX,(t)) - e(xJ}, t L 0,

where gdx) = max,s,,k(,,)Ihfl(x) - hfl(x,)l, and put Z, = a(V.). It follows from (9.5) and the continuity of e that Z, * 2, where 2 is defined in terms of X as Z, is in terms of X, . Therefore Z,( T ) 3 Z( T ) for all T B D(Z), and for such T

I s l b N

144 CONVaGENCEoFflOUIlllTYkYAPUlET

by Theorem 3.1, where the last inequality uses the fact that

(9.9) sup min {g,(x) - dx,)} < 0. x e r i s i s N

Let q > 0, let T > 0 be as above, let m 2 I , and choose a compact set r0,, c E such that

(9.10) f{x(r) E r0,, for 0 s r s T} z 1 - ~ 2 - ~ ' ~ ;

this is possible by Theorem 5.6, Lemma 2.1, and Remark 6.4. By (9.8), there exists n, 2 1 such that

(9.1 1)

Finally, for n = 1,. . . , n, - 1, choose a compact set r,, ,,, c E such that

(9.12) P{X,,(r) E r!!: for 0 s r s T} 2 1 - @-".

Letting r, = UZ;' r,,,, we have

(9.13) inf P{X,(r) E r:', for 0 5 t 5 T} 2 1 - $-",

so if we define r,, (being complete and totally bounded) and

(9.14)

inf P{X,(r) E r;!: for 0 s t s 7') 2 1 - $-". n z l r

nz I

to be the closure of n,,,,, r;'"', then T,, r is compact

inf P{X,(t) E T,, r for 0 s r 5 T } 2 1 - q. 012 I

Finally, we note that

(9.15) (el A a I , gk 0 X,*(gl AaIr. . . rgk hak) 0 X

for all gl, . . . , gk E H and aI, . . . ,ak E R. This, together with the fact that If is dense in c ( E ) in the topology of uniform convergence on compact sets, allows one to conclude that the finite-dimensional distributions converge. The details are left to the reader. 0

9.3 Corollary Let E be locally compact and separable, and let Ed be its one-point compactification. If {X, ) is a sequence of processes with sample paths in Ds[O, 00) and if { s o X,} is relatively compact for everyfE e(€) (the space of continuous functions on E vanishing at infinity), then {X,} is relatively compact considered as a sequence of processes with sample paths in DEIIO, 00). If, in addition, (X,,(rl), . . . , Xm(rk)) *(X(fI), . . . , X(tk)) for all finite subsets { t , , . . . , t r } of some dense set D c [O, a), where X has sample paths in DEIO, a), then X, =+ X in DEIO, w).

9. FURTMR CRITERIA FOR RELATIVE COMPACTNESS IN D,# a01 145

Proof. I f f € C(E*), then /( a ) - J ( A ) restricted to E belongs to e(&). Conse- quently, ( / o X , } is relatively compact for every f E C(EA), and the relative compactness of ( X , } in D,A[O, 00) follows from Theorem 9.1. Under the additional assumptions, X , * X in DEA[O, 00) by Theorem 7.7, and hence X , * X

0 in &[O. ao) by Corollary 3.2.

We now consider the problem of verifying the relative compactness of (10 X,} , where/€ c ( E ) is fixed. First, however, recall the notation and terminology of Section 7 of Chapter 2. For each a, let X , be a process with sample paths in D,[O, 00) defined on a probability space (0,. 4ca, Pa) and adapted to a filtration (.at:}. Let Ip, be the Banach space of real-valued (S:}-progressive processes with norm II Y II = sup,,o E[l Y(r)I] c 00. Let

(Y, 2) E 9, x 9,: Y( t ) - 1 Z(s) ds is an {.%:}-martingale

and note that completeness of (9;) is not needed here.

9.4 Theorem Let ( E , r ) be arbitrary, and let { X , ) be a family of processes as above. Let C, be a subalgebra of C(E) (e.g., the space of bounded, uniformly continuous functions with bounded support), and let D be the collection pf J E C ( E ) such that for every E > 0 and T > 0 there exist (c, 2,) E .d, with

and

(9.18) sup E[ll2,~Ip, < 00 for some p E (1. a]. a

(IIhIt,,,=CS,Tlh(r)tPdtI1” if P < a; IIhII,,r=esssup,,,,,th(r)/.) If C,, is contained in the closure of D (in the sup norm), then { J o X,} is relatively compact for each /E C,; more generally, {u,, . . . ,Jk) 0 XJ is relatively compact in DAIIO, 00) for allJ,,/2,. . . ,f,, E C,, 1 5 k 5 00.

9.5 Remark (a) Taking p = 1 in condition (9.18) is not suficient. For example, let n 2 1 and consider the two-state (E = (0,l)) Markov process X, with infinitesimal matrix

(9.19) (-: -:)

146 CONVRGENCE OF PROMIlUTV kwua

and P(X,(O) 3: 0) = 1. Given a function f on (0, I) , put Y. = f 0 X , and 2, = ( A , f ) 0 X,, so that (Y., Z,,) E 2, and

(9.20) ECIA~I(Xa(t))ll

= nVO -f(I)IP{xa(t) = 1) + If(l) -f(OIIp{xa(l) a 0)

= If([) -f(O)l(l + (n - Inn + ])-'(I - e-("+l)'))

5 211(1) -f(O)ls

for all t 2 0. However, observe that the finitedimensional distributions of X , converge to those of a process that is identically zero, but {X,} does not converge in distribution.

(b) For sequences {X, } , one can replace sup, in (9.17) and (9.18) 7

by 0

la CONV~CENCE TO A mocm H c,p, 00) 147

Let l / p + l / q = 1 and l/p' + I/q' = I, and note that fi+'lh(s)l ds s PI/ h I t p , f + for 0 s r 5 T. Therefore, if we define

(9.26) r.(4 = 2 SUP IJ2(X,(4) - WI ~ c ( O . T + l l n O

+ 411I'II SUP lf(X,(d) - K(4l

+ 6 '"'It z a II pa, r + 1 + 2 II f II 6 ' " II za II p . r + 1 9

I a (0. f + I I n Q

then

(9.27) €CU(xa(t + u)) - . f ( ~ A t ) ) ) 2 IFJI 1; ECya(6) ICI. Note that this holds for all 0 s r r; T and 0 s u 9 S (not just for rational I and u) by the right continuity of X,. Since

(9.28) sup ECv.@)l S (2 + 4IIfIIb a

+ 8'"' SUP ECitz&iIp*.T+rI + 211JIIJ1" SUP ECIIZaIIp, r + i J a

we may select E depending on 6 in such a way that (8.29) holds. Therefore, { f ~ X,} is relatively compact by Theorem 8.6 (see Remark 8.7(a)).

Let I 5 k -c oo. Given Jl,. . . ,fk E C,, define v',((s) as in (9.26) and set ydd) = x= v'.cS). Then

for 0 s t 5; T and 0 s u 5 6, and the yi(6) can be selected so that (8.29) holds. Finally, relative compactness for k = Q, follows from relative compactness for all k < 00. (See Problem 23.) 0

10. CONVERGENCE TO A PROCESS IN C,lO,ao)

Let (E, r) be a metric space, and let C,[O,oo) be the space of continuous functions x: [O, 00) --+ E. For x e DEIO, oo), define

(10.1)

where

(10.2)

4 x 1 = e-"[J(x , u)A 13 du,

J(x , u) = sup r(x(t), X(l-)). o s t s r

Since the mapping x-+ J(x, . ) from Da[O, 00) into Dfo, m,[O, 00) is continuous (by Proposition 5 3 , it follows that J is continuous on Da[O,oo). For each x rs DJO, 00). J(x, * ) is nondecreasing, so J(x) = 0 if and only if x E C,[O, 00).

118 CONVERCENCE OF ROBAULITY M E W I E S

10.1 Lemma Suppose (x ,} c DECOY ao), x E Ds[O, o), and d(xa, x) = 0. Then

(10.3) lim sup r(x,(r), x(r)) s J(x, u) - a-m Osrsu

for all u 2 0.

Let c(D,[O, a), d,) be the space of bounded, real-valued functions on Ds[O, 00) that are continuous with respect to the metric

(10.5) ddx, y) = e-' sup CdxW, f i t ) ) / \ 11 du,

that is, continuous in the topology of uniform convergence on compact subsets of [0, ao). Since d 5 dU , we have YE ES L€l(D,[O, a), d ) c CS(D,[O, a), du).

O S l S U

10.2 Theorem Let X,, n = 1, 2,. .., and X be processes with sample paths in DEIO, a), and suppose that X , 310 X. Then (a) X is a.s. continuous if and only if J(X,) - 0, and (b) if X is as. continuous, then f ( X , ) r*f(X) lor every 9,-measurable f E C(D,[O, ao), d,,).

Proof. Part (a) follows from the continuity of J on DEIO, 00). By Lemma 10.1, if {x,} c D d O , 00). x E Cs[O, ao), and limn-- d(x,, x) = 0,

then lima-,mdu(xa, x) 5 0. Letting F c W be closed and j be as in the statement of the theorem,/-'(F) is d,-closed. Denoting its d-closure by/-, i t follows that f- A CJO, 00) = f -'(F) n CEIO, ao). Therefore, if P, * P on (DEW, 001, d ) and P(Cd0, 4) r= 1,

(10.6) lim P , / - I ( F ) < lim P,UTO) S If-) a- m a-rm

= WT by Theorem 3.1, so we conclude that P, f - * PJ-'. This implies (b).

c,m 00)) = w-W) n CJO, a)) = P f - W

0

10. CONVERGENCE TO A noass IN c , ~ , a01 149

The next result provides a useful criterion for a proccss with sample paths in D,[O, ao) to have sample paths in C,[O, 00). It can also be used in conjunc- tion with Corollary 8.10.

10.3 Proposition Let (E, r) be separable, and let X be a process with sample paths in DEIO, 00). Suppose that for each T > 0, there exist > 0. C > 0, and 8 > 1 such that

(10.7)

whenever 0 5 s I t 5 T, where 9 = r A 1. Then almost all sample paths of X belong to C,[O, ao).

ECqYX(t), m)l s C@ - S)O

Proof. Let T be a positive integer and observe that

(10.8)

By Fatou's lemma and (10.7), the right side of (10.8) has zero expectation. and hence the left side is zero 8.5. 0

2.1

q'(X(t), X ( t - ) ) s - lim 1 qU(X(k2-") , X( (k - 1)2-")). O < f S T n + w h a 1

10.4 Proposition For n parameter R'-valued process, let a,, > 0, and define

1, 2,. . . , let {l-#), k = 0, 1,. . .) be a discrete-

(1 0.9) Xn(t) = UCantl)

and

(10.10) zn(t) = V.(CantI) + (ant - CanrlMV.(Can 11 + 1) - K(Ca,tl))

for all t 2 0. Note that X, has sample paths in D , [ O , 00) and Z, has sample paths in Cw[O, 00). If limn-.w a, = OD and X is a process with sample paths in CRd[O, 00). then X, * X if and only if Z, * X.

Proof. We apply Corollary 3.3. It suffces to show that, if either X , = X or 2, * X , then d(X,, Z,)+ 0 in probability. (The two uses of the letter d here should cause no confusion.) For n = 1,2,. . . ,

(10.11) d(X , , Z,) 5 1 e-' sup (IX,( t ) - X,(r-)IA I ) du 0 st s Y + 0"- I

J(X,). I;

and

150 CONVERGENCE OF ?ROBAUUTY Muwlss

provided a;' I; E. But the function J,: Du[Ol a)-, CO, 11, defined for each E>Oby

(10.13) JXx) = [ e-' sup

is continuous and satisfies lim,,,JL(x) = J(x) for all x E Dnr[Ol 00). Conse- quently, if 2, - X, then (10.12) and Theorem 3.1 imply that

(10.14) lim P{J(x,,) 2 S } s Iim lim P{J,(z,) 2 6)

s lim P{J,(X) 2 6) = 0

sup (Ix(t) - x(s)lA 1) du ( 2 J(x)), O S f S U f - S A f $ S e f

- n - a 8-40 n-tu

r - 0

for all S > 0. so we conclude that J(X,,)- 0 in probability. The same conclusion follows from Theorem 10.2 if X, + X. In either case, (10.1 1) implies that

0 d(X,, Z,>+ 0 in probability, as required for Corollary 3.3.

11. PROBLEMS

1. Let (S, d) be complete and separable, and let P, Q E P(S). Show that there exists p c M(P, Q) (see Theorem 1.2) such that

and show that p2(P, Q) 5 I{ P - Q 11 s 3p(P, Q). H i m : Recall that $ f dP = f ) / f l l P { f z r} df if f 2 0, and note that II(e - d( a , F))VOJIBL 5 1 for 0 < E c 1.

3. Show that f l S ) is separable whenever S is.

J. Suppose (P,,) c P(R), P E LP(R), and P, PO P. Define

(1 1.3) G,,(x) = inf { y E W: P,(( - 00, yl) 2 x}

and (I 1.4)

for 0 < x < 1, and let t: be uniformly distributed on (0, I). Show that GJC) has distribution Pn for each n, G(<) has distribution P, and limn- G,( e) = G(C) as.

a x ) = inf { y E R: P(( - co, y] ) 2 x)

11. PROOlEMS 151

5. Let (S, d) and (S’, d’) be separable. Let X,, n = 1, 2 ,..., and X be S-valued random variables with X, * X. Suppose there exist Bore1 measurable mappings h, , k = l , 2,. . . , and h from S into S’ such that:

(a) For k = 1, 2, . . . , hk is continuous a.s. with respect to the distribution of x .

(b) hk -+ h as k -+ 00 a.s. with respect to the distribution of X .

(c) limkem P{d(hk(X, ) , h(X,)) > E ) = 0 for every E > 0. Show that h(X,) * h(X) . (Note that this generalizes Corollary 1.9.)

Let X,, n = I , 2,. . . , and X be real-valued random variables with finite second moments defined on a common probability space (Q, 9, P). Suppose that {X,] converges weakly to X in L2(P) (i.e.,

ECX, Z ] = E C X Z ] , 2 E L?(P)), and (X,} converges in distribution to some real-valued random variable Y. Give necessary and sumcient conditions for X and Y to have the same distribution.

Let X and Y be S-valued random variables defined on a probability space (Q, 9, P). and let Y be a sub-a-algebra of 9. Suppose that M c c(S) is separating and

( I 1.5)

for everyjE M. Show that X = Y a.s.

Let M = { f ~ c(W): f has period N for some positive integer N } . Show that M is separating but not convergence determining.

Let M c c(S) and suppose that for every open set G c S there exists a sequence {f,} c M with 0 <f, 5 xG for each n such that bp-limn..mf, = xG . Show that M is convergence determining.

10. Show that the collection of all twice continuously Frechet differentiable functions with bounded support on a separable Hilbert space is convergence determining.

11. Let S be locally compact and separable. Show that M c e(S) is convergence determining if and only if M is dense in &) in the supremum norm.

-

I

6.

7.

N - / ( X ) I g1 = J( Y )

8.

9.

12. Let x e DEIO, 00). and for each n L I , define x, E DEIO, m) by %,(I) = x(( [nt] /n) A n). Show that lim,,+m d(x,, x) = 0.

13. Let E and F be metric spaces, and let/: E - . F be continuous. Show that the mapping x -f 0 x from D,[O, 00) into D,[O, 00) is continuous.

14. Show that O,[O, m) is separable whenever E is.

152 CONMRCENCE Of ROBMIUTY MEASURES

15. Let (E, r) be arbitrary, let r, c r2 c * - . be a nondecreasing sequence of compact subsets of E, and define

(11.6) S = {x E D,[O, 00): x(t ) E r, for 0 s r I n, n = 1, 2 ,... }. Show that (S, d) is complete and separable, where d is defined by (5.2).

16. Let (E, r) be complele. Show that if A is compact in DEIO, a), then for each T > 0 there exists a compact set Tr c E such that x( t ) E TT for 0 s t s Tand all x E A.

17. Prove the following variation of Proposition 6.5. Let {x,} c DE[O, 00) and x E DEIO, a). Then d(x,, x) = 0 if and

only if whenever t, 2 s, L 0 for each n, r 2 0, limm-.ms, = I, and r , = t , we have

(1 1 *7) lim Mxn(tnh ~ ( t ) ) V Hxn(sn), x(t))I A dxn(sm)v x(t - )) = 0 n - m

and

(1 1.8) lim r(x,,(t.l, x ( 0 ) A Cr(xn(tn)s x(t - 1) V r(Xn(Sn1, x(r -))I = 0.

18. Let (E , t) be complete and separable. Let {X,} be a family of processes with sample paths in DEIO, OD). Suppose that for every E > 0 and T > 0 there exists a family { X: '} of processes with sample paths in D,[O, GO)

(with X; and X, defined on the same probability space) such that

sup P sup ~-(x:~(t) , Xu(r)) 2 e < e ( 1 1.9)

and {X: r } is relatively compact. Show that {X,} is relatively compact.

19. Let (E , r) be complete and separable. Show that if {Xu} is relatively compact in DEIO, a), then the compact containment condition holds.

20. Let {N,} be a family of right continuous counting processes (i.e., N , ( O ) = O , N , ( t ) - N , ( t - ) = 0 or 1 for all t > O ) . For k = O , 1 , ..., let T; = inf ( t 2 0: N,(t) 2 k } and A; = r; - T ; - , (if T:-, < 00). Use Lemma 8.2 to give necessary and sufficient, conditions for the relative compactness of {N,}.

21. Let (E, r ) be complete and separable, and let {X,} be a family of processes with sample paths in &[O, 00). Show that {Xs} is relatively compact if and only if for every E > 0 there exists a family {X:) of pure jump processes (with X: and X, defined on the same probability space) such that sup, ~ u p , ~ ~ r ( X # ) , X&)) < E a.s.. (X:( t ) ) is relatively compact for each rational f z 0, and (N:) is relatively compact, where N$) is the number of jumps of X: in (0, t ] .

I- ID

I I a O S I S ~

11. ROllLEMS 153

(a) Give an example in which {X,) and { 5) are relatively compact in D,[O, 00). but {(X,, K)} is not relatively compact in D,,[O, 00).

(b) Show that if {X,}, { X}, and {X, + K) are relatively compact in Da[O, 00). then {(X,, Y,)) is relatively compact in Dwl[O, 00).

(c) More generally, if 2 5 r < 00, show that {(X: , X:, . . . , Xi)} is relatively compact in DJO. 00) if and only if {X:} and {X: + Xi} (k , 1 = 1,. . . , r) are relatively compact in DJO, 00).

Show that { ( X i , X.', . . .)} is relatively compact in D,,,[O, co) (where R" has the product topology) if and only if {(Xi , . . . ,Xi)) is relatively compact in D,,[O, 00) for r = I , 2,. . . . Let ( E , t ) be complete and separable, and let {X,) be a sequence of processes with sample paths in DEIO, 00). Let M be a subspace of C(€) that strongly separates points. Show that if the finite-dimensional distributions of X, converge to those of a process X with sample paths in D,[O. 00). and if { g 0 X,} is relatively compact in DJO, 00) for every g E M, then X, * X .

Let ( E , r ) be separable, and consider the metric space (C,[O, 00). d"), where d , is defined by (10.5). Let 9 denote its Bore1 a-algebra. (a) For each t 2 0, define n,: CEIO, a)---) E by R,(x) = x(f). Show that

A7 = ~ ( n , : 0 -< r < a). (b) Show that d , determines the same topology on C,[O, 00) as d (the

latter defined by (5.2)). (c) Show that CJO, 00) is a closed subset of D,[O, 00), hence i t belongs

to .YE, and therefore .9 c .YE.

(d) Suppose that { P,) t .9(D,[O, to)), P E 9YDE[0, 00)). and P.(C,[O, ce)) = P(C,[O, 00)) = I for each i t . Define {Q,} c .P(C,[O, 00)) and Q E SyC,[O, 00)) by Q, = Pela and Q = P la. Show that P, P on D,[O, 00) if and only if Q, * Q on CJO, a)).

Show that each of the following functions jk : D,[O, 00)- D,[O, 00) is continuous :

22.

23.

24.

25.

26.

( I 1.10)

27. Let .d c .@(S) be closed under finite intersections and suppose each open

154 CONVERGENCE OF PROBMIUTV MEASURES

set in S is a countable union of sets in d. Suppose P, P , E 9 ( S ) , n = 1, 2,. . . , and lime-* P,,(A) = P ( A ) for every A E 1. Show that P ,

28. Let (S, d ) be complete and separable and let d c dit(S). Suppose for each closed set F and open set U with F c U, there exists A E d such that F c A c U. Show that if { P , } c 9(S) is relatively compact and limn-m P,(A) exists for each A E d, then there exists P E 9(S) such that Pn =$. P.

P.

12. NOTES

The standard reference on the topic of convergence of probability measures is Billingsley’s (1968) book of that title, where additional historical remarks can be found.

As originally defined, the Prohorov (1956) metric was a symmetrized version of the present p. Strassen (1965) noticed that p is already symmetric and obtained Theorem 1.2. Lemma 1.3 is essentially due to Dudley (1968). Lemma 1.4 is a modification of the marriage lemma of Hall (1935), and is a special case of a result of Artstein (1983). Prohorov (1956) obtained Theorem I .7. The Skorohod (1956) representation theorem (Theorem 1.8) originally required that (S, d) be complete; Dudley (1968) removed this assumption. For a recent somewhat stronger result see Blackwell and Dubins (1983). The continuous mapping theorem (Corollary 1.9) can be attributed to Mann and Wald (1943) and Chernoff( 1956).

Theorem 2.2 is of course due to Prohorov (1956). Theorem 3.1 (without (a)) is known as the Portmanteau theorem and goes

back to Alexandroff (1940-1943); the equivalence of (a) is due to Prohorov (1956) assuming completeness and to Dudley (1968) in general. Corollary 3.3 is called Slutsky’s theorem.

The topology on Ds[O, a) is Stone’s (1963) analogue of Skorohod’s (1956) J , topology. Metrizability was first shown by Prohorov (1956). The metric J is analogous to Billingsley’s (1968) do on D[O, I]. Theorem 5.6 is essentially due to Kolrnogorov (1956).

With a different modulus of continuity, Theorem 6.3 was proved by Proho- rov (1956); in its present form, it is due to Billingsley (1968).

Similar remarks apply to Theorem 7.2. Condition (b) of Theorem 8.6 for relative compactness is due to Kurtz

(1975)’ as are &he results preceding it in Section 8; Aldous (1978) is responsible for condition (c). See also Jacod, Memin, and Metivier (1983). Theorem 8.8 is due to ChenEov (1956).

The results of Section 9 are based on Kurtz (1975). Proposition 10.4 was proved by Sato (1977). Problem 5 is due to Lindvall(1974) and can be derived as a consequence of

Theorem 4.2 of Billingsley (1968).

4

In this chapter we study Markov processes from the standpoint of the generators of their corresponding semigroups. In Section I we give the basic definitions of a Markov process, its transition function, and its corresponding semigroup, show that a transition function and initial distribution uniquely determine a Markov process, and verify the important martingale relationship between a Markov process and its generator. Section 2 is devoted to the study of Feller processes and the properties of their sample paths, and Sections 3 through 7 to the martingale problem as a means of characterizing the Markov process corresponding to a given generator. In Section 8 we exploit the characterization of a Markov process by its generator (either through the determination of its semigroup or as the unique solution of a martingale problem) to give general conditions for the weak convergence of a sequence of processes to a Markov process. Stationary distributions are the subject of Section 9. Some conditions under which sums of generators characterize Markov processes are given in Section 10.

Throughout this chapter E is a metric space, M ( E ) is the collection of all real-valued, Bore1 measurable functions on E, W E ) c M(E) i s the Banach space of bounded functions with 11/11 = sup,.,If(x)l, and Q E ) c E ( E ) is the subspace of bounded continuous functions.

155

GENERATORS AND MARKOV PROCESSES



156 GENERATORS AND M M K O V PROCESSES

1. MARKOV PROCESSES AND TRANSITION FUNCTIONS

Let {X( t ) , t r 0} be a stochastic process defined on a probability space (n, SF, P) with values in €, and let 9: = o(X(s): s 5 t). Then X is a Markov process if

(1.1) P{x(t + S) E rr4F:J = f{x( t + S) E rlx(t)) for all s, t L 0 and r E 1(E). If {Yl} is a filtration with 9: c Y,, t 2 0, then X is a Markoo process with respect to (Y,} if (1.1) holds with gF: replaced by Y,. (Of course if X is a Markov process with respect to {gJ, then i t is a Markov process,) Note that (1.1) implies

(1.2)

for all s, 2 0 andfc B(E).

transitiori function if

(1.31 PO, x, * E 69, 0. 4 E LO, 00) x E,

(1.4) P(0, x, 9 ) = 6, (unit mass at x), x E E,

ELT(X(t + 4) I 93 = E U ( X 0 + 4) I X(01

A function P(t, x , r) defined on [0, ao) x E x g(E) is a time homogeneous

(1 3)

(1.6) ~ ( t + s, x, r) = P ( ~ , y , r)p(t, X, dy),

P( ., ’, r) E BKO, 00) x E). r E W E ) ,

and

S, t 2 0, x E E, r E q ~ ) . 5 A transition function P(t, x, r) is a transition function for a time-

homogeneous Markoo process X if

(1.7) P { X ( C + S) E r 1 P:} = P(S, x ( t ) , r) for all s, i 2 0 and r E A?(E), or equivalently, if

(1.8)

for all s, r 2 0 andftz B(E).

assumption given (1.7). observe that (1.7) implies

ELT(X(t + s))l9t,“I = If(YIP(S, X(t) , dY)

To see that (1.6). called the Chapman-Kolmogorov property, is a reasonable

(1 -9) ~ ( t + S, x(u), r) = P ( X ( U + t i- S) E rl sp,”} = E C P I X ( ~ + ~ + ~ ) € ~ I S ” , X , , } I ~ , Y I

= E [ P ( S , X ( U + t), r)i*:] = s P ( s . Y, W Y r . X(UX dY)

for all s, t , u 2 0 and r E B(E).

1. MARKOV PROCESSES AND TRANSITION FUNCTIONS 157

The probability measure v E B(E) given by v(r) = P ( X ( 0 ) E r} is called the

A transition function for X and the initial distribution determine the finite- initial distribution of X .

dimensional distributions of X by

(1.10) P { X ( O ) E To. X(t , ) E r , , . . . , X(r , ) E r,,]

P(tn - t n - 1 , Y n - 1 1 r n ) P ( t n - I - f m - 2 3 Y n - z r dyn - 1 ) = 1" . . . s,-,

' f'(l1, yo ~ Y I )iu(dyo).

In particular we have the following thcorem.

1.1 Theorem Let P(r, x, r) satisfy (1.3Hl.6) and let v E B ( E ) . I f for each c 2 0 the probability measure P ( f , x, . )u(dx) is tight (which will be the case if (E, r ) is complete and separable by Lemma 2.1 of Chapter 3), then there exists a Markov process X in E whose finite-dimensional distributions are uniquely determined by ( I . 10).

proof. For I c [O, a), let E' denote the product space IlaE , E, where for each s, E, = E, and let 8, denote the collection of probability measures defined on the product a-algebra ~ J c , 4 f ( E J ) . For s E I, let X(s) denote the coordinate random variable. Note that nSE, 4f(E,) = a(X(s): s E I) .

Let {si, i = 0, I , 2 , . . .) c [O, 00) satisfy .Ti # s, for i # j and so = 0, and fix x, E E. For n > 1, let 1 , < t 2 < . * * < r , be the increasing rearrangement of s l , . . . , s,. Then i t follows from Tulcea's theorem (Appendix 9) that there exists P, E P,sil such that P,(X(O) E To, X ( r , ) E TI,. . . , X(r , ) E T,} equals the right side of (1.10) and P,{X(s , ) = xo) = I for i > n. Tulcea's theorem gives a measure Q. on E'" * ' ' . * snl. Fix xo E E and define P , = Q, x S,=,,, * O . , , on &I = E ( S 1 .....Sd Eh*I.Jm+Z.... I. The sequence { P, ) is tight by Proposition 2.4 of Chapter 3, and any limit point will satisfy (1.10) for { I , , . . . , t , ) c (si}. Consequently {P , , } converges weakly to P"'I E P,,,,.

By Problem 27 of Chapter 2 for E E 9 f ( € ) I 0 . m' there exists a countable subset { s i ) c [O, 00) such that B E a(X(si): i = 1, 2,. . .), that is, there exists B E si?(€)lSii such that B = {(X(sl), X ( s 2 ) , . . .) E b} . Define P(5) = Plr4)(b). We leave it to the reader to verify the consistency of this definition and to show

0 that X, defined on (Elo* m), 9(E)I0 - m! P), is Markov.

Let P , denote the measure on ,W(E)ro* R'' given by Theorem 1 . 1 with v = 6,, and let X be the corresponding coordinate process, that is. X ( t , w ) = d r ) . I t follows from (1.5) and (1.10) that P,(B) is a Bore1 measurable function of x for f?=(x(O)~r,, ...., X(t, ,)~r, ,} , O < r l < r z < . . - < t , , r,, I- ,,...,

E B(E). In fact this is true for all 5 E 9?(E)Io, m'.

158 GENERATORS AND MARKOV PROCESSES

1.2 Proposition Let P, be as above. Then P,(B) is a Borel measurable function of x for each B E O(E)Io*

Proof. The collection of B for which P,(B) is a Borel measurable function is a Dynkin class containing all sets of the form ( X ( 0 ) E ro, . . . , X(t,) E r,} E

0 @E)ro* and hence all B E B(E)l0* m! (See Appendix 4.)

Let (Y(n) , n = 0, 1, 2,. . .} be a discrete-parameter process defined on (Q, 9, P) with values in E , and let 9: = a(Y(k): k s n). Then Y is a Markoil chain if

(1.11) P { Y(n + m) E ri9:) = f{ Y(n + m) E ri Y(n)]

for all m, n 2 0 and r E 1 ( E ) . A function p(x* r) defined on E x A#(@ is a rransition function if

(1.12) P(X , ' 1 E w3, x E E,

and

(1.13)

A transition function p(x, r) is a transirion function for u time-homogeneous Markou chain Y if

(1.14) P { Y ( n + 1) E rp=,Y} = p ( ~ ( n ) , r), 0, r E a(@. Note that

* . cl(Y(n), ~ Y A . As before, the probability measure v E P(E) given by v(T) = P { Y(0) E r} is called the initial distribution for Y. The analogues of Theorem 1.1 and Propo- sition 1.2 are left to the reader.

Let { X ( t ) , t ;r 0}, defined on (n, 9, P), be an E-valued Markov process with respect to a filtration ('3,) such that X is {Yl}-progressive. Suppose f ( t , x. r) is a transition function for X, and let r be a ('J,}-stopping time with 7 < co as. Then X is strong Markou at r if

(1.16) PWT + t ) E r I 9,) = f i t , XW, r) for all t 2 0 and l-' E a ( E ) , or equivalently, if

(1.17)

for all t 2 0 andf6 B(&). X is a strong Murkou process with respect to (3,) if X is strong Markov at T for all (y,}-stopping times T with T c a0 as.

1. MARWOV PUOCEJMS AND TRANSITION FUNCTIONS 159

1.3 Proposition Let X be E-valued, {Yl}-progressive, and {Yl}-Markov, and let P(t, x, r) be a transition function for X. Let T be a discrete (91)-slopping time with r < a, as. Then X is strong Markov at r .

Proof. then B n ( T = t i } E Y,, and hence for t 2 0,

Let t, , t , , . . . be the values assumed by r, and let /E B(E). If B E gr,

Summing over i we obtain

for all B E Y,, which implies (1.17). 0

1.4 Remark Recall (Proposition 1.3 of Chapter 2) that every stopping time is the limit of a decreasing sequence of discrete stopping times. This fact can frequently be exploited to extend Proposition 1.3 to more-general stopping times. See, for example, Theorem 2.7 below.

1.5 Proposition Let X be E-valued, {Y,}-progressive, and {Y,)-Markov, and let P(t, x, r) be a transition function for X . Let t be a {Y,}-stopping time, suppose X is strong Markov at T + t for all t 2 0, and let B E $#(E)ro. m). Then

Proof. First consider B of the form

(1.21) {X E E'O. m): x(t i ) E r,, i = I , . . . , n }

where 0 s r , 5 t , I; * . s t,, r,, r2,. . . , I-,, E g ( E ) . Proceeding by induction on n, for B = { x E Elo* m': x ( t , ) E r,} we have

(1.22) P { x ( ~ + .)E BIY,) = P { x ( ~ + t , ) E r,IE4,} = Or,, X ( r ) , r,) by (l.l6), but this is just (1.20). Suppose now that (1.20) holds for all B of the

160 GENERATORS AND U K O V ?ROC€SSES

form (1.21) for some fixed n. Then

for 0 S t l 5 c 2 s * * * s rn and& E RE). Let B be of the form (1.21) with n + 1 in place of n. Then

Ordinarily, formulas for transition functions cannot be obtained, the most notable exception being that of one-dimensional Brownian motion given by

(1.25) 1

P(r, x, r) = J- 2nr exp { - e} 2r dy.

Consequently, directly defining a transition function is usually not a useful method of specifying a Markov process. In this chapter and in Chapters 5 and 6, we consider other methods of specifying processes. In particular, in this chapter we exploit the fact that

(1.26)

1. MAIKOV PROCESSES AND TRANSITION FUNCTIONS 161

defines a measurable contraction semigroup on E ( E ) (by the Chapman- Kolmogorov property (1 A)).

Let (Tct)} be a semigroup on a closed subspace L c B(E). With reference to (M), we say that an E-valued Markov process X corresponds to { T(t ) } if

for all s, I L 0 a n d f e L. Of course if {T(t)) is given by a transition function as in (1.26), then (1.27) is just (1.8).

1.6 Proposition Let E be separable. Let X be an €-valued Markov process with initial distribution v corresponding to a semigroup { T(r)] on a closed subspace L c B(€) . I f L is separating, then { T( t ) } and v determine the finite- dimensional distributions of X.

Proof. ForfE L and t 2 0, we have

= ~C7WS(x(o))l = W ) S ( x ) W ) . 5 Since L is separating, u,(T) EE P ( X ( t ) E r} is determined. Similarly i f f € L and g E WE), then for 0 s t , < t z r

J

and the joint distribution of X(r,) and X ( t , ) is determined (cf. Proposition 4.6 of Chapter 3). Proceeding in this manner, the proposition can be proved by induction. 0

Since the finite-dimensional distributions of a Markov process are determined by a corresponding semigroup (T( t ) } , they are in turn determined by its full generator A^ or by a suficiently large set A c A’. One of the best approaches for determining when a set is “suficiently large” is through the martingale problem of Stroock and Varadhan, which is based on the observation in the following proposition.

162 CENEMTORS AND W K O V PROcpssEs

1.7 Proposition Let X be an E-valued, progressive Markov process with transition function P(t, x, r) and let {T(t)} and A^ be as above. If (A 8) B A^ then

(1 30) 4 0 = f (X(l1) - C' B ( W ) ds

is an (S':}-rnartingale.

Proof. For each I , u 2 0

(1.31) E C W + u ) l m

= ~ / W P ( u ,

0

We study the basic properties of the martingale problem in Sections 3-7.

2. MARKOV jUMP PROCESSES AND FELLER PROCESSES

The simplest Markov process to describe is a Markov jump process with a bounded generator. Let p(x, r) be a transition function on E x @(E) and let 1 E B(E) be nonnegative. Then

(2.1) A m 4 = r2(x) { M Y ) - - J ( X ) ) A X . dY)

defines a bounded linear operator A on B(€), and A is the generator for a Markov process in E that can be constructed as follows.

(2.3) X(t) = \

and note that (2.1) can be rewritten as

A Y(O),

Y(k) , -!!.L I t < A

0 5 L e 0 4 Y(0)) ’

L - I k

j= 0 4 yo” j = o 4 Y O ) ) ’

Let { Y’(k), k = 0, I , . . .} be a Markov chain in E with initial distribution v and transition function p’(x, r), and let Y be an independent Poisson process with parameter 1.. Define

(2.4) xyt) = Y’(Y( t ) ) , t 2 0.

We leave it to the reader to show that X and X’ have the same finite- dimensional distributions (Problem 4). 0 bserve that

P ! ( X ) = / (Y)P’ (X , dv) I (2.7)

defines a linear contraction P on B(E) and that, by (2.5). A = A(P - I). Conse- quently, the semigroup { 7’(t)) generated by A is given by

LetfE B(E). By the Markov property of Y‘(cf. (1.15)).

(2.9) ECS( Y’(k + 0) I Y’(0). . . . , YYOJ = P’YI Y’(N

164 GENERATORS AND MMKOV PROCESSES

for k , 1 = 0, 1,. . ., and we claim that

(2.10) ECS(Y'(k + W)) I s,1 = fWX'(r))

-- P(A n { Y( t ) = I } ) J Pf(Y'(I)) dP B

Pj(X(t)) dP. -s A n B n (V(c )= I )

Since ( A n B n { V(t ) = I } : A E Sy, B E S:; I = 0, 1, 2, . . .} is closed under finite intersections and generates ,Fl, by the Dynkin class theorem (Appendix 4) we have

If( Y ( k + V(t))) dP = pL/(X'(r)) dP (2.13)

for all A E 9, , and (2.10) follows, Finally, since Y has independent increments, I

(2.14) ECf(X'(t + 4) I s 1 1

= ECf( y'( w + 4 - W) + W) I9,l

for all s, t 2 0. Hence X' is a Markov process in E with initial distribution v corresponding to the semigroup { T(t)} generated by A.

We now assume E is locally compact and consider Markov romses with

uous functions vanishing at infinity with norm 11 f 11 .C sup, I /(x) I . Note semigroups that are strongly continuous on the Banach space E (E) of contin-

2. MARKOV JUMP mocEssEs AND FELLER mocEssEs 165

that e ( E ) = C(E) if E is compact. Let A C E be the point at infinity if E is noncompact and an isolated point if E is compact, and put Ed = E u (A); in the noncompact case, E A is the one-point compactification of E. We note that if E is also separable, then EA is metrizable. (See, for example, pages 201 and 202 of Cohn (1980).)

A semigroup (T(t)] on C(E) is said to be positiue if T(t) is a positive operator for each t L 0. (A positive operator is one that maps nonnegative functions to nonnegative functions.)

An operator A on e(&) is said to satisfy the positioe maximum principle if wheneverfe 9 ( A ) , xo E E, and supxc E f ( ~ ) = f ( x o ) 2 0, we have Af(xo) I 0.

2.1 lemma the positive maximum principle is dissipative.

Let E be locally compact. A linear operator A on C ( E ) satisfying

Proof. Let J E O(A) and 1 > 0. There exists xo E E such that I f ( x o ) l = 11 f 1 1 . Suppose f ( x , ) 2 0 (otherwise replace f by -f). Since supxe E / ( ~ ) = f ( x o ) 2 0, AJ(x,) 5 0 and hence

We restate the Hille-Yosida theorem in the present context.

2.2 Theorem Let E be locally compact. The closure A of a linear operator A on C(E) is single-valued and generates a strongly continuous, positive, contraction semigroup { T(t)} on d(E) if and only if:

(a) 9 ( A ) is dense in e ( E ) . (b) A satisfies the positive maximum principle. (c) - A ) is dense in c ( E ) for some 2. > 0.

Proof. The necessity of (a) and (c) follows from Theorem 2.12 of Chapter I . As for (b), i f / € 9 ( A ) , xo E E, and supxrEf(x) = J ( x o ) 2 0, then

for each t 2 0, so Af(x,) 5 0. Conversely, suppose A satisfies (a)+). Since (b) implies A is dissipative by

Lemma 2.1, A’ is single-valued and generates a strongly continuous contraction semigroup (T( t ) } by Theorem 2.12 of Chapter 1. To complete the proof, we must show that { T(r)} is positive.

ia GENERATORS AND MMKOV moasss

Let/€ 9(A) and d > 0, and suppose that inf,,,J(x) < 0. Choose {h} c 9 ( A ) such that ( A - A)f,-+ (A - J ) f , and let x, E E and xo E E be points at whichi, andf, respectively, take on their minimum values. Then

(2.17) inf, &I - A ) f ( x ) s & ( A - A)-/&,) n-. a0

s limAj&c,)

= Jf (X0)

< 0,

I--

where the second inequality is due to the fact that inf,,,.&(x) =f,(x,,) 5; 0 for n suficiently large. We conclude that iffe 9(A) and A > 0, then (A - A)/z 0 impliesf2 0, so the positivity of (T(t)) is a consequence of Corollary 2.8 of Chapter 1. 0

An operator A c W E ) x B(E) (possibly multivalued) is said to be conservative if ( 1 , 0) is in the bp-closure of A. For example, if (1, 0) is in the full generator of a measurable contraction semigroup (T(t)}, then T(t)l = 1 for all f 2 0. and conversely. For semigroups given by transition functions, this property is just the fact that f ( t , x, E) = 1.

A strongly continuous, positive, contraction semigroup on e ( E ) whose generator is conservative is called a Feller semigroup, Our aim in this section is to show (assuming in addition that E is separable) that every Feller semigroup on e ( E ) corresponds to a Markov process with sample paths in DECOY ao). First, however, we require several preliminary results, including our first convergence theorem.

2.3 lemma Let E be locally compact and separable and let {T(t)} be a strongly continuous, positive, contraction semigroup on t ( E ) . Define the operator TA(t) on C(EA) for each t 2 0 by

(2.18) TA(t)f = fa) + W ( j - f(Ah (We do not distinguish notationally between functions on EA and their restrictions to E.) Then { TA(t)} is a Feller semigroup on C(EA).

Proof. It is easy to verify that (Tqr) } is a strongly continuous semigroup on C(EA). Fix t z 0. To show that TA(f) is a positive operator, we must show that if a E R, S E e ( E ) , and a + / r 0, then a + T(t)fr 0. By the positivity of T(t), T ( t ) ( f + ) 2 0 and T ( t ) ( f - ) 2 0. Hence - T(c)/s T ( f ) ( f - ) , and so ( T ( t ) f ) - 5 T(t)( f -). Since T(t) is a contraction. ll T ( t # / - ) II s l l f - JI 5 a. Therefore (T(t)f)- $ a, so a + T(t)f2 0.

Next, the positivity of TA(c) gives I TA(t)/I 5 TA(t) II f II 5 II f II for all / E C(EA), so 11 Tyr) 11 = 1. Finally, the generator A” of (T*(t)) clearly contains (1. 0). 0

1. M M K O V jUMP MOCESSES A N D FELLER MOCESSES 167

2.4 Proposition Let E be locally compact and separable. Let {T( t ) ) be a strongly continuous, positive, contraction semigroup on (?((E), and define the semigroup (TA( t ) } on C(EA) as in Lemma 2.3. Let X be a Markov process corresponding to {TA( t ) } with sample paths in D,,[O, a), and let i = inf{t 2 0 : X ( r ) = A or X(r -) = A}. Then

(2.19) P{t < ao, X(T + s) L- A for all s 2 0) = P{t < 00).

Let A be the generator of { T(f)} and suppose further that A is conservative. If P { X ( 0 ) E E } = 1, then P { X E D,[O, a)} = I .

Proof. Recalling that Ed is metrizable, there exists g E C(E') with g > 0 on E and g(A) = 0. Put/= e-'TA(u)g du, and note that/> 0 on E andf(A) = 0. By the Markov property of X, (2.20) E[e- ' f (X( t ) ) l .aF,"t] = e-'TA(t - s ) j ( X ( s ) )

= e-s ~~;- 'TA(uMX(s)) du

5 e-Y(X(s)) , 0 s s < t ,

so e - ' f (X( t ) ) is a nonnegative {.Ff+}-supermartingale. Therefore, (2.19) is a consequence of Proposition 2.15 of Chapter 2. It also follows that P { X ( t ) = A} = P{t s t ) for all r 2 0.

Let A' denote the generator of { TA(r)}. The assumption that A is conservative (which refers to the bp-closure of A in B(E) x B(E)) implies that ( x E , 0) is in the bp-closure of A' (considering Ad as a subspace of B(E') x B(E*)). Since the collection of (J g) E B(Eh) x B(E') satisfying

(2.2 I )

is bp-closed and contains A', for all t 2 0 we have

(2.22)

and if P{ X ( 0 ) E E} = I, we conclude that P ( X E DEIO, 00)) = P ( t = 00) = 1.

P{r > I } = P { X ( t ) E E } = P { X ( O ) E E } ,

0

A converse to the second assertion of Proposition 2.4 is provided by Corol- lary 2.8.

2.5 Theorem Let E be locally compact and separable. For n = I , 2, ... let {T , ( t ) ) be a Feller semigroup on (?(El, and suppose X n is a Markov process corresponding to { T&)} with sample paths in Dd[O, 00). Suppose that { T(t)} is a Feller semigroup on e ( E ) and that for each/€ e(E), (2.23)

168 CENauTORS AND MARKOV PROCESSES

If {XJO)} has limiting distribution u E @E), then there is a Markov process X corresponding to {T(t)} with initial distribution u and sample paths in DEIO, a), and X, * X.

Proof. For each n 2 1, let A, be the generator of { X ( t ) } . By Theorem 6.1 of Chapter I , (2.23) implies that for each/€ 9 ( A ) , there exist/; E 9 ( A n ) such that /,+/and A,f,-, Af. Sincef,(X,(t)) - Po A,L(X,,(s)) ds is an (.FaF,X"}-martingale for each n 2 1, and since 9 ( A ) is dense in e ( E ) , Chapter 3's Corollary 9.3 and Theorem 9.4 imply that ( X J is relatively compact in D,,[O, 00).

We next prove the convergence of the finite-dimensional distributions of {X,}. For each n 2 1, let {Tt(r)} and {T"(t)} be the semigroups on C(EA) defined in terms of {T(r)) and { T(r)} as in Lemma 2.3. Then, for each/€ C(EA) and t 2 0.

(2.24) lim ECf(xn(t))l = lim ECT3t)j(Xn(O))I n-m n - a

= I TA(l)f oc)v(dx)

by the Markov property, the strong convergence of {T3r)}, the continuity of TA(r)J and the convergence in distribution of {XAO)}. Proceeding by induction, let M be a positive integer, and suppose that

(2.25)

exists for allJ,, . . . , fm E C(EA) and 0 s I , < - * * < r,. Then

(2.26) lim ECfI(Xn(t1)) . . * f m ( X S t m ) ) f m + t ( X a ( t m + 1))l n-m

= lim EC-f1(Xn(t 1)) * - f m ( X n ( t m ) ) T % t m + 1 - tm)fm + I ( x n ( t m ) U n+ OD

= lim W"(Xn(f 1)) * * * Sm(xn(tm))TA(tm + 1 - t m ) f m + I (XAtm) ) I n-m

existsforallf,, ...,f,+, cC(E")andOst , < - - * < t , , , + , . It follows that every convergent subsequence of (X,} has the same limit, so

there exists a process X with initial distribution v and with sample paths in DEd[O, GO) such that X, =5 X. By (2.26), X is a Markov process corresponding to { T4(t)}. so by Proposition 2.4, X can be assumed to have sample paths in DEIO, 00). Finally, Corollary 9.3 of Chapter 3 implies X n r+ X in DEIO, 00). 0

2.6 Theorem p,(x, r) be a transition function on E x &(E) such that T., defined by

Let E be locally compact and separable. For n = 1, 2, ... let

(2.27)

2. MADKOV lUMP PROCESSES AND FELLER FROCESSLS 169

satisfies Let E, > 0 satisfy lirn,-m E, = 0 and suppose that for everyfE e ( E ) ,

: e((E)--+ e((E). Suppose that { T(r)} is a Feller semigroup on C(E).

(2.28)

For each n 2 1, let { Y,(k), k = 0, 1. 2, . . .} be a Markov chain in E with transition function p,(x, r), and suppose { Y,(0)} has limiting distribution v E P ( E ) . Define X , by X, ( t ) E V.([t/&,J). Then there is a Markov process X corresponding to (T(t)} with initial distribution v and sample paths in D,[O, a), and X,= X.

Proof. Following the proof of Theorem 2.5, use Theorem 6.5 of Chapter I in place of Theorem 6.1. 0

2.7 Theorem Let E be locally compact and separable, and let { T(t)f be a Feller semigroup on d(&). Then for each v E 9+(E), there exists a Markov process X corresponding to (T( t ) ) with initial distribution v and sample paths in DEIO, 00). Moreover, X is strong Markov with respect to the filtration 9, = *:+ = n t > o * : + , .

Proof. Let n be a positive integer, and let

(2.29) A, = A( / - n - ' A ) - ' = n[( / - n - ' ~ ) - ' - /I be the Yosida approximation of A. Note that since ( I - n-'A)--' is a positive contraction on (?(El, there exists for each .Y E E a positive Borel measure p,(x, I-) on E such that

(2.30)

for allJE e ( E ) . I t follows that p,,(., r) is Borel measurable for each I- E . g ( E ) . For each ( A g) E A, (2.30) implies

(1 - n - '4 - !f(x) = S ( y k . ( x . d y ) s (2.31)

Since the collection of ( J , g) E B(E) x E(E) satisfying (2.31) is bp-closed, i t includes ( I , 0) and hence p,,(x, E ) = I for each x E &. implying that p,(x. r) is a transition function on E x .g(E). Therefore, by the discussion at the beginning of this section, the semigroup (T, , ( t )} on c(&) with generator A, corresponds to a jump Markov process X, with initial distribution v and with sample paths in

Now letting n - r 00, Proposition 2.7 of Chapter 1 implies that for each T , ( t ) / = T(t)f; so the existence of X follows from

DECO. 00).

J E C(E) and t 2 0, Theorem 2.5.

170 GENERATORS AND M M K O V PROCfsSES

Let r be a discrete (g,}-stopping time with T < 00 as. concentrated on ( I , , r 2 , ...}. Let A E Y,, s > 0, and JE e ( E ) . Then A n { s = t i } E .%:+& for every E > 0, so

(2.32) 1 / ( X ( r + s)) d P = 1 AX(t, + s)) df A n l t - 1 1 1 A n l r ~ C r l

= T(s - E)f(X(t, + E ) ) dP A n l r =ti)

for 0 < E s s and i = I, 2, ... . Since {T(t)} is strongly continuous, T(s)f is continuous on E, and X has right continuous sample paths, we can take E = 0 in (2.32). This gives

(2.33) ECS(X(T + s)) I f s r l = T(s)f(x(~))

for discrete r. If r is an arbitrary {S,}-stopping time, with r < 00 as., it is the limit of a

decreasing sequence { q,} of discrete stopping times (Proposition 1.3 of Chapter 2), so (2.33) follows from the continuity of T(s)/on E and the right continuity of the sample paths of X. (Replace 7 by T,, in (2.33), condition on Y,, and then let n-+ el.) 0

2.8 Corollary Let E be locally compact and separable. Let A be a linear operator on c(€) satisfying (a)-@) of Theorem 2.2, and let { T(t)} be the strongly continuous, positive, contraction semigroup on c ( E ) generated by 2. Then there exists for each x E E a Markov process X, corresponding to (T(t)} with initial distribution 6, and with sample paths in Ds[O, 00) if and only if A is conservative.

Proof. The sufficiency follows from Theorem 2.7. As for necessity, let {g,,} c &(I - A ) satisfy bp-lim,,,g, = 1, and define {f,J c 9 ( A ) byl; = ( I - A)-'g, . Then

for all x E €, so bp-lim,,,f, = 1 and bp-limn-.m Af, = bp-lim,,, ( J - QrJ = 0. 0

We next give criteria for the continuity of the sample paths of the process obtained in Theorem 2.7. Since we know the process has sample paths in DEIO, ao), to show the sample paths are continuous it is enough to show that they have no jumps.

2. MARKOV IUMP PROCESSES AND FELLER PROCESSES 171

2.9 Proposition Let (E, r) be locally compact and separable, and let { T(r)} be a Feller semigroup on c ( E ) . Let P(t, x, r) be the transition function for (T(r)} and suppose for each x E E and E > 0,

(2.35) lim r - 'P ( r , x , B(x, E ) C ) = 0. l-0

Then the process X given by Theorem 2.7 satisfies P ( X E C,[O, a)) = I .

2.10 Remark Suppose A is the generator of a Feller semigroup {T(r)} on C ( E ) with transition function P(t, x, r), and that for each x E E and E > 0 there exists f E 9 ( A ) with f ( x ) = )I f 11, supy, B,x. rF f ( y ) M < )I f 11, and AJ(x) = 0. Then (2.35) holds. To see this, note that

(2.36) ( I1 J II - W P ( t , X. B(x. E ) C ) SJ(x) - E,CS(X(l)) l

= - [ T ( s ) A / ( x ) ds.

Divide by r and let t --• 0 to obtain (2.35). 0

Proof. Note that for each x E E and 2: 0,

(2.37) T ; T i i P ( t , y , E ( y , ~ ) C ) s l i m Y 'X Y-X

For each S > 0 there is a r(x, S) s 6 such that for t = f(x, 6) the right side of (2.37) is less than Bt(x, 8). Consequently, there is a neighborhood U , of x such that y E U, implies

(2.38) W x , a), y, B(y, E)C) s 2 6 0 , 6).

Since any compact subset of E can be covered by finitely many such U , , we can define a Bore1 measurable function s(y, S ) s 6 such that

(2.39)

and for each compact K c E

f W Y . 4, Y, B(Y, 47 5 2 W y , 6).

(2.40) inf Iy, b) > 0. Y ~ K

Define to = 0 and

(2.41) TII + I = TII + ~ ( ~ ( T I I ) , a). Note that limk+m?k = 00 since ( X ( s ) : s 5 t ) has compact closure for each r 2 0.

Let n - I

(2.42)


and observe that n - 1

(2.43) iic Nd(n) - p(s(X(rh), x(rh), B(x(rk) , &r) h - 0

is a martingale. Let K c E be compact, let T > 0, and define

Then by the optional sampling theorem

s E[25~,] 5 2 4 T + 5).

Finally, observe that limd,, N&) = 1 on the set where X has a jump of size larger than E before T and before leaving K. Consequently, with probability one, no such jump occurs. Since E, T, and K are arbitrary, we have the desired result. 0

We close this section with two theorems generalizing Theorems 2.5 and 2.6. Much more general results are given in Section 8, but these results can be obtained here using essentially the same argument as in the proof of Theorem 2.5.

2.11 Theorem Let E, E l , E , , ... be metric spaces with E locally compact and separable. For n = 1, 2, ,. ., let 4, : En-+ E be measurable, let (q(f)f be a semigroup on WE,) given by a transition function, and suppose V, is a Markov process in En corresponding to (T,(t)} such that X , = rtl, 0 Y,, has sample paths in D,[O, 00). Define n, : B(E)-t B(E,) by n , j = j o , q , , (cf. Section 6 of Chapter 1). Suppose that { T(r)} is a Feller semigroup on e ( E ) and that for each / E e ( E ) and f 2 0, T,(t)n,$-+ T(f)f (i.e., il T,(r)n,f- rr, T(t)f I1 -+ 0). If {X,(O)} has limiting distribution v E sP(E), then there is a Markov process X corresponding to (T(r)} with initial distribution v and sample paths in O,[O, a)), and X, * X .

3. THE MARTINGALE PROBLEM: CENERALl~ES AND SAMPLE PATH PROPERTIES 173

Finally, we give a similar extension of Theorem 2.6.

2.12 Theorem Let E, E , , E l , . . . be metric spaces with E locally compact and separable. For n = I, 2, . .., let q,, : En--+ E be measurable, let p,(x, r) be a transition function on E , x 4?(E,), and suppose { Y,(k), k = 0, I , 2, . . .} is a Markov chain in En corresponding to p , (x , r). Let E, > 0 satisfy limfi-.m, c, = 0. Define X n ( f ) = Vn( Yn([(Ct/&nJ))*

(2.47) T J ( X ) = j / (y)pn(x* dy), SE B(En),

and R,: E(E) -+ B(E,) by n, f = f 0 q,. Suppose that { T(t)) is a Feller semigroup on C(E) and that for each , f ~ e ( E ) and t >. 0, T!,"em~~,J'--+ T(t)J If (X,(O)) has limiting distribution v f 9(E), then there is a Markov process X corresponding to { T(t)} with initial distribution v and sample paths in DEIO, a), and X , * X.

3. SAMPLE PATH PROPERTIES

THE MARTINGALE PROBLEM: GENERALITIES A N D

In Proposition 1.7 we observed that, if X is a Markov process with full generator A, then

is a martingale for all (J g) E 4. In the next several sections we develop the idea of Stroock and Varadhan of using this martingale property as a means of characterizing the Markov process associated with a given generator A. As elsewhere in this chapter, E (or more specifically (E, r)) denotes a metric space. Occasionally we want to allow A to be a multivalued operator (cf. Chapter I , Section 4), and hence think of A as a subset (not necessarily linear) of W E ) x B(E). By a solution of the martingale problemfor A we mean a measurable stochastic process X with values in E defined on some probability space (n, 9, P ) such that for each (J g) E A, (3.1) is a martingale with respect to the filtration

Note that if X is progressive, in particular if X is right continuous, then '9: = F:. In general, every event in *9(x differs from an event in 9: by an event of probability zero. See Problem 2 of Chapter 2.

If (9,) is a filtration with Y, 3 *9: for all r 2 0, and (3.1) is a (9,)-martingale for all (5 g) E A, we say X is a solurion o j the miartingale

174 CENEMTOIS AND W K O V PRocEssfs

problem for A with respect to (Y,]. When an initial distribution p e 4yE) is specified, we say that a solution X of the martingale problem for A is a solution of the martingale problem for (A, p) if PX(0)-' = p.

Usually X has sample paths in &[O, 00). It is convenient to call a probability measure P E. 9(DE[O, 03)) a solution of the martingale problem for A (or for (A , p)) if the coordinate process defined on (&LO, oo), Y E , P) by

(3.3) X(t , O) 5 4 t h w E D,y[O, OO), t 2 0,

is a solution of the martingale problem for A (or for (A, p)) as defined above.

for A if and only if Note that a measurable process X is a solution of the martingale problem

whenever 0 5 t , < t z < * * 4 < I,+ I , (f, g) E A, and h,, . . . , h, E B(E) (or equivalently e(E)). Consequently the statement that a (measurable) process is a solution of a martingale problem is a statement about its finite-dimensional distributions. In particular, any measurable modification of a solution of the martingale problem for A is also a solution.

Let A, denote the linear span of A. Then any solution of the martingale problem for A is a solution for As. Note also that, if A") c A('), then any solution of the martingale problem for A'2) is also a solution for A('), but not necessarily conversely. Finally, observe that the set of pairs (I; g) for which (3.1) is a {fB,}-martingale is bp-closed. Consequently, any solution of the martingale problem for A is a solution for the bp-closure of As. (See Appendix 3.)

3.1 Let A"' and A"' be subsets of S(E) x HE). If the bp- closures of (A"))s and (A'''), are equal, then X is a solution of the martingale problem for A"' if and only if it is a solution for A").

Proposition

Proof. This is immediate from the discussion above. 0

The following lemma gives two useful equivalences to (3.1) being a martingale.

3.2 lemma Then for fixed 1 E R, (3.1) is a {Y,}-martingale if and only if

Let X be a measurable process, Y, 3 *9:, and let j ; g E B(E).

(3.5)

3. THE MARTINGALE PROBLEM: GENERALITIES A N D SAMPLE PATH PROPERTIES 175

is a {Y,}-martingale. If inf,/(x) > 0, then (3.1) is a {5fl]-martingale if and only if

(3.6)

is a {Y,}-martingale.

Proof. If (3.1) is a (Y,}-martingale, then by Proposition 3.2 of Chapter 2 (see Problem 22 of the same chapter),

- l g ( X(s ) ) ds e - I' - g( X( u)) du Re - Is ds sbl = e-'YY(X(r)) + e-"[Af(X(s)) - g(X(s))] ds Jrb

is a {4/,)-martingale. (The last equality follows by Fubini's theorem.) I f inf,/(x) > 0 and (3.1) is a {Y,)-martingale, then

176 GENERATORS AND W K O V PUOCES-

is a {91)-martingale. The converses follow by similar calculations (Problem 14). a

The above lemma gives the following equivalent formulations of the martingale problem.

3.3 Proposition Let A be a linear subset of B(E) x B(E) containing (I , 0) and definc

(3.9) A + = ((1; 8) E A : inf,f(x) > 0).

Let X be a measurable E-valued process and let '3,a *.F',". Then the following are equivalent:

(4 X is a solution of the martingale problem for A with respect to (Cg,} ,

(b) X is a solution of the martingale problem for A + with respect to

(c) For each (1; g) E A, (3.5) is a (9,)-martingale. (d) For each (J 8) E A + , (3.6) is a {91}-martingale,

{Cg I } .

Proof. Since (A'), = A, (a) and (b) are equivalent. The other equivalences follow by Lemma 3.2. 0

For right continuous X, the fact that (3.5) is a martingale whenever (3.1) is, is a special case of the following lemma.

3.4 Lemma Let X be a measurable stochastic process on ( R , 9 , P) with values in E. Let u, u : [0, 00) x E x R-, R be bounded and a[O, 00) x a(E) x $-measurable, and let w : [0, a)) x [O, 00) x E x R-, R be bounded and B[O, 00) x S[O, 00) x a(E) x 9-measurable. Assume that u(t, x, w) is continuous in x for fixed t and w, that u(t, X(r) ) is adapted to a filtration {Y,}, and that u(t, X(r)) and W(t, I, X(r)) are {44,}-progressive. Suppose further that the conditions in either (a ) or (b) hold:

(a) For every r1 > t , 2 0,

and

3. THE MARTINGALE PROIMLEM: GENERALITIES A N D SAMRE PATH PROPERTIES 177

Moreover, X is right continuous and

(3.12)

(b)

lim E[I w(t - 6, t , X ( t ) ) - w(r, r , X ( t ) ) l ] = 0, t z 0. d + O +

For every t 2 > t , 2 0,

and

Moreover, X is left continuous and

(3.15) lim E[ I w(r + 6, t , X(r)) - w(t, t , X(r ) ) I ] = 0, t 2 0. & d o +

Under the above assumptions,

f, (3.16)

is a {Y,}-martingale.

Proof. we have

Fix I, > t , 2 0. For any partition f , = so < s, < s, < . . < s, = /, ,

= E [ ~ : * { u ( s , X ( S ' ) ] + w(s", S, X(.S))] ds q,, I 1


Clearly, only dissipative operators arise as generators of Markov processes. One consequence of Lemma 3.2 is that we must still restrict our attention to dissipative operators in order to have solutions of the martingale problem.

3.5 Proposition Let A be a linear subset of WE) x B(E). If there exists a solution X , of the martingale problem for (A, 6,) for each x E E, then A is dissipative.

Proof. Given (S, g) E A and d > 0, (3.5) is a martingale and hence

As stated above, we usually are interested in solutions with sample paths in D,[O, m). The follcwing theorem demonstrates that in most cases this is not a restriction.

3.6 Theorem Let E be separable. Let A c c ( E ) x B(E) and suppose that 9 ( A ) is separating and contains a countable subset that separates points. Let X be a solution of the martingale problem for A and assume that for every E > 0 and T > 0, there exists a compact set K , ,such that

(3.22) P { X ( t ) E Kt, ,. for all t E [O, T ] n QJ > 1 - E.

Then there is a modification of X with sample paths in D,[O, 00).

Proof. Let X be defined on (Q, 9, P). By assumption, there exists a sequence {(fi, 8,)) c A such that {jJ separates points in E. By Proposition 2.9 of Chapter 2, there exists R' c Q with P(W) = I such that

(3.23) J;(W) - s"...)) ds 0

has limits through the rationals from above and below for all r 2 0, all i, and all w E R'. By (3.22) there exists 0" t Q' with P(Q") = I such that { X ( r , w): r E [O, T3 n Q} has compact closure for all T > 0 and w E Q". Suppose w E W. Then for each f 2 0 there exist s, E Q such that s, > 1, lime.+*sn = t , and lim, X ( s , , w ) exists, and hence (3.24)

3. THE MARTlNCAL€ PROBLEM: C€N€RALtTW AND S W € tATH PRO?ERNES 179

where the limit on the right exists since w E R'. Since {/I} separates points we have

(3.25) lim X ( s ) 3 Y ( t ) *-.I+

S € Q

exists for all t 2 0 and w E R". Similarly

(3.26) lim X(s) = Y - ( t ) 1-1 - 3.Q

exists for all t > 0 and w E R", so Y has sample paths in D,[O, 00) by Lemma 2.8 of Chapter 2.

(3.27)

Since X is a solution of the martingale problem, if follows that

ECS( Y( t ) ) I 9:1 = lim ~ C S ( X ( S ) ) I 9:1 = S ( W ) 3 - 1 + S S Q

for every/€ 9 ( A ) and t 2 0. Since 9 ( A ) is separating, P ( Y ( t ) = X ( t ) ) = I for 0 all r 2 0. (See Problem 7 of Chapter 3.)

3.7 Corollary Let E be locally compact and separable. Let A c e(€) x B(E) and supposc that 9 ( A ) is dense in e(€) in the norm topology. Then any solution of the martingale problem for A has a modification with sample paths in DEb[O, 00) where E A is the one-point compactification of E.

and A" = Then any solution of the martingale problem for A considered as a process with values in EA is a solution of the martingale problem for A".

0 Since A" satisfies the conditions of Theorem 3.6, the corollary follows.

In the light of condition (3.22) and in particular Corollary 3.7, i t is sometimes useful to first prove the existence of a modification with sample paths in D,[O, 00) (where i!? is some compactification of E) and then to prove that the modification actually has sample paths in DEIO, 00). With this in mind we prove the following theorem.

3.8 Theorem Let (k, r ) be a metric space and let A c E ( E ) x E(&. Let E c E be open, and suppose that X is a solution OC the martingale problem for A with sample paths in &[O, 00). Suppose ( x E , 0) is in the bp-closure of A n (C(&) x I?(&). If P(X(0) E E} = 1. then P ( X E D,[O, 00)) = 1.

180 GENERATORS A N D MARKOV PROCESSES

Proof.

(3.29)

Then r , s t2 s B - E if and only if lim,-.m t, E T $ t . For (f, g) E: A n (c(8) x B(&)),

For m = 1, 2, . . ., define the {P:+)-stoppIng time

t, = inf t : inf,,.g-&r(y, X( t ) ) < - . i in 7 * and lirnm+m X(7,Ac) s Y(r) exists. Note that Y(r) is in

(3.30)

is a right continuous {Sf}-martingale, and hence the optional sampling theorem implies that for each r L 0,

(3.31) E[f(x(rm A r ) ) ~ = EC/(X(O))I + .[ s" 'g(x(S)) ds]. 0

Letting m -+ ao, we have

(3.32) W ( Y ( t ) l l = ECf(X(O))J + E[J'^'B(X(S)) 0 ds],

and this holds for all (f, g ) in the bp-closure of A n (&!?) x @)). Taking LA d = (XE ? O), we have

(3.33) P { r > t ) = P{ Y(r) E E } = 1, t 2 0.

Consequently, with probability 1, X has no limit points in fi - E on any bounded time interval and therefore has almost all sample paths in DJO, 00).

n 3.9 Proposition Let i!?, A, and X be as above. Let E c A!? be open. Suppose there exists { ( f n 9 g")} c A n (c@) x B(k)) such that

(3.34)

(3.35)

and {g,} converges pointwise to zero. If' P(X(0)e E) = 1, then e { X E

U,[O, m)} = 1.

Proof. Substituting (f.' gn) in (3.32) and letting n-+ a, Fatou's lemma gives

(3.36) P{Y( t ) E E ) 2 P(X(0) E E } = 1. 0

3.10 Proposition Let A!?, A , and X be as above. Let E , , E l , ... be open subsets of k alid let E =: nk E k . Suppose ( x S , 0) is in the bp-closure of A n (e(& x BIkU. If P{X(O) E E} = 1, then P { X B OSLO, 00)) = 1.

3. THE MARTINGALE PROlLEM: GENERALITIES A N D SAMPLE PATH PROPERTIES 181

Proof. Let rk be defined as in (3.29) with E replaced by E,. Then the analogue of (3.32) gives

(3.37) P(lim,,,X(r~At) E E,} 2 P{ l imm+mX(7~At ) E E } = I .

Therefore almost all sample paths of X are in DJO, no) for every k , and hence in DJO, a). 0

3.11 In the application of Theorem 3.8 and Propositions 3.9 and 3.10, E might be locally compact and tf? = EA, or E = n, F, , where the F, are

0

Remark

locally compact, i? = nk F,d, and E, = n , < k F , x n,,k F f .

We close this section by showing, under the conditions of Theorem 3.6, that any solution of the martingale problem for A with sample paths in D,[O, 00) is quasi-left conrinuous, that is, for every nondecreasing sequence of stopping times r, with limw-m r, = r .c co a.s., we have limn-.m X(r,,) = X ( r ) as .

3.12 Theorem Let E be separable. Let A c C(E) x B(E) and suppose 6 ( A ) is separating. Let X be a solution of the martingale problem for A with respect to {Y,}, having sample paths in D,[O, 00). Let T~ s T~ s . . be a sequence of (9,)-stopping times and let r =

(3.38)

In particular, P(X(t) = X(t-)} = 1 for each t > 0.

r,,. Then

P lim X(rn) = X(r), t -= or) = P { r 4 00).

L + m 1 Proof. Clearly the limit in (3.38) exists. For ( J g) E A and I 1. 0,

(3.39) limJ(X(T,, A t ) ) = lim E I - m n-ag

= E[S(X(r A t ) ) I v 59J9

and (3.38) follows. (See Problem 7 of Chapter 3.)

n

0

3.13 Corollary Let (E, r ) be separable, and let A and X satisfy the conditions of Theorem 3.12. Let F c E be closed and define r = inf { t : X(t) E F or X ( t - ) E F ) and d = inf { t : X(r ) 6 F). (Note that u need not be measurable.) Then r = d as .

Proof. Note that (r = 0 ) = { r -= 00, X(r) E F ) u {r = 0 0 ) . Note that by the right continuity of the martingales, X is a solution of the martingale problem for A with respect to {9:+}. Let U, = {y : in fXeF r(x. p) < l /n), and define

182 CENEMTORS AND MARKOV ?ROCESSES

r,, = inf {c: X(r) E Urn). Then r,, is an {.Ff+}-stopping time, r , s r2 i; * . and JimaMm T,, = T. Since X ( T J E on, Theorem 3.12 implies

(3.40) T -= a, X ( r ) 5: lim X(r,,) E F 0 n-ao

4. THE MARTINGALE PROBLEM: UNIQUENESS, THE MARKOV PROPERTY, AND DUALITY

As was observed above, the statement that a measurable process X is a solution of the martingale problem for (A, p) is a statement about the finite- dimensional distributions of X . Consequently, we say that uniqueness holds for solutions of the martingale problem for (A, p) if any two solutions have the same finite-dimensional distributions. If there exists a solution of the martingale problem for ( A , p) and uniqueness holds, we say that the martingale problem for (A, p ) is well-posed. If this is true for all p B rp(E), then the martingale problem for A is said to be well-posed. (Typically, if the martingale problem for (A, 6,) is well-posed for each x E E, then the martingale problem for (A, p ) is well-posed for each p E .P(E). See Problems 49 and 50.) We say that the martingale problem for (A, p) is well-posed in DEIO, a) (C,[O, a)) if there is a unique solution P E 9 ( D E [ 0 , a)) (P d 9(C,[O, ao))). Note that a martingale problem may be well-posed in DEIO, 03) without being well-posed, that is, uniqueness may hold under the restriction that the solution have sample paths in DEIO, a) but not in general. See Problem 21. However, Theorem 3.6 shows that this difficulty is rare. The following theorem says essentially that a Markov process is the unique solution of the martingale problem for its generator.

4.1 Theorem Let E be separable, and let A c B(E) x B(E) be linear and drssipative. Suppose there exists A' c A, A' linear, such that A?(A - A') = 9") = L for some 1 > 0, and L is separating. Let p E 9 ( E ) and suppose X is a solution of the martingale problem for (A, p). Then X is a Markov process corresponding to the semigroup on L generated by the closure of A', and uniqueness holds for the martingale problem for (A, p).

Proof. Without loss of generality we can assume A' is closed (it is single- valued by Lemma 4.2 of Chapter I ) and hence, by Theorem 2.6 of Chapter I , it generates a strongly continuous contraction semigroup { T(t)} on L. In particular, by Corollary 6.8 of Chapter I ,

4. THE MARnNCAlE PROOLLM: UNIQVENESS, THE MARKOV PROERTV, AND DUAClTV 163

We want to show that

for all$E L, which implies the Markov property, and the uniqueness follows by Proposition 1.6.

I f (1; 8) E A' and R > 0, then (3.5) in Lemma 3.2 is a martingale and hence

/ ( X ( t ) ) = € [ [ m e - a s ( A ~ ( X ( t + s)) - g(X( t + s))) ds 0

(4.3)

which gives

for all h E L. Iterating (4.4) gives

(4.5) (I - n- 'A ' ) -kh(X( t ) )

Suppose h E 9(A'). Then

(4.6) (I - n- 'A')-"'"lh(X(r))

= E[h(X(t 4- u)) I s':]

The second term on the right is bounded b:,

184 GENERATORS AND MARKOV PROcEsIps

where the A, are independent and exponentially distributed with mean 1. Consequently (4.7) goes to zero as n 4 m, and we have by (4.1)

(4.8) T(u)h(X(t)) = lim ( I - n-'A')-"%(X(t)) a-m

= E[h(X(r + 24)) 193. - Since g ( A ' ) = L, (4.2) holds for allfc L.

Under the conditions of Theorem 4.1, every solution of the martingale problem for A is Markovian. We now show that uniqueness of the solution of the martingale problem always implies the Markov property.

4.2 Theorem Let € be separable, and let A c H E ) x B(E). Suppose that for each p E P(E) any two solutions X, Y of the martingale problem for (A , p) have the same one-dimensional distributions, that is, for each t > 0,

(4.9) P{x(t) E r-} = P { Y ( C ) E r}, r E a(q. Then the following hold.

(a) Any solution of the martingale problem for A with respect to a filtration {Y,) is a Markov process with respect to {Y,}, and any two solutions of the martingale problem for ( A , p ) have the same finite- dimensional distributions (i.e., (4.9) implies uniqueness).

(b) If A c c ( E ) x B(E), X is a solution of the martingale problem for A with respect to a filtration (9,}, and X has sample paths in DEIO, m), then for each a.s. finite (g,}-stopping time T,

(4. i 0)

for allJrs B(E) and t 2 0. (c) If, in addition to the conditions of part (b), for each x E E there

exists a solution P, E @(DEIO, 00)) of the martingale problem for (A, 6,) such that PJB) is a Bore1 measurable function of x for each B E 9, (cf. Theorem 4.6), then, defining T(c)f(x) = J(w(t))P,(dw),

(4.1 I ) JW (X(7 + 0) I 3J = TldJ(x(d) for all f E B(€), t 2 0, and as. finite {g,}-stopping times T (i.e., X is strong Markov).

ECf (X(T f 0) I %I = E C J ( W + 0) I X(d1

Proof. Let X , defined on (0, qC, P), be a solution of the martingale problem for A with respect to a filtration {Y,), fix I 2 0, and let F E qr satisfy P(F) > 0. For B E 9 define

(4.12)

4. THE MARTINGALE PROILEM: UNIQUENESS, THE MARKOV PROPERTY, AND DUALITV 185

and

(4.13)

and set Y ( - ) = X ( r + .). Note that

(4.14) p I { Y ( 0 ) E r} = p 2 ( Y ( 0 ) E r} = P ( X ( r ) E r I F } .

With (3.4) in mind we set

(4.16)

and similarly for E , [ q ( Y ) ] . Consequently, Y is a solution of the martingale problem for A on (a, 9. PI) and (a, 6, P2). By (4.9), E , [ j ( Y ( t ) ) ] = EZ[f( Y(t ) ) ] for eachfE 8 ( E ) and t 2 0, and hence

(4.17) ECXF E C f ( x ( r + I ) ) I 9,Il

Since F E 9, is arbitrary, (4.17) implies

(4.1 8)

which is the Markov property. Uniqueness is proved in much the same way. Let X and Y be solutions of

the martingale problem for ( A , p) defined on (a, 9, P ) and (r, 9, Q) respectively. We want to show

ECXF EC/(X(r + t ) ) I X ( r ) J l .

E C f ( X ( r + 0) I9,I = W - ( X ( r + t ) ) I .Wl,

(4.19)

for all choices of t k E [O, 00) andf, E B(E) (cf. Proposition 4.6 of Chapter 3). I t is suflicient to consider only/, > 0. For m = 1, (4.19) holds by (4.9). Proceeding by induction, assume (4.19) holds for all m 5 n, and f ix 0 5 I , < t , < . + . < I, andf,, .... fm E W E ) , / , > 0. Define

(4.20)

(4.2 1)

786 CENMTORE AND M M K O V CQocEsSpJ

and set &t) = X(r, + c ) and f'(t) = Y(r, + t). By the argument used above, 8 on (Q, 9, p) and Pon (r, g, Q) are solutions of the martingale problem for A. Furthermore, by (4.19) with m = n,

(4.22)

= EQCf ( WNI, /E m3,

t 2 0, f E B(4.

so ,? and ? have the same initial distributions. Consequently, (4.9) applies and

(4.23)

As in (4.22). this implies E'Cf (x(t))l = EQCS(P(NI,

and, setting I,, , 3: t, + t , we have (4.19) for m = n + 1. For part (b), assume that A c c(E) x W E ) and that X has sample paths in

DEIO, 03). Then (3.1) is a right continuous martingale with bounded increments for all (f, g) E A and the optional sampling theorem (see Problem 11 of Chapter 2) implies (4.25) E C M T + 9) I 3,l = 0.

so (4.10) follows in the same way as (4.18). Similarly for part (c), if F E Y, and f l F ) > 0, then

(4.26)

and

(4.27)

define solutions of the martingale problem with the same initial distribution D and (4. I 1) follows as before.

Since it is possible to have uniqueness among solutions of a martingale problem with sample paths in DEIO, 00) without having uniqueness among solutions that are only required to be measurable, it is useful to introduce the terminology DJO, m) martingale problem and C,[O, m) martingale problem to indicate when we are requiring the designated sample path behavior.

4.3 Corollary Let E be separable, and let A c B(E) x B(E). Suppose that for each p E @(EX any two solutions X , Y of the martingale problem for (A, p)

4. THE MARTINGALE PROBLEM: UHQUEMSS, T M MARKOV R M R T V , AND DUALllY 187

with sample paths in DEIO,oo) (respectively, CEIO, 00)) satisfy (4.9) for each r 2 0. Then for each 1.1 E *E), any two solutions of the martingale problem for (A, p) with sample paths in &[O, 00) (C,[O, 00)) have the same distribution on DE[o* m, (CECO, 00)).

proof. Note that 2 and defined in the proof of Theorem 4.2 have sample paths in D,[O, 03) (CEIO, m)) if X and Y do. Consequently, the proof that X and Y have the same finite-dimensional distributions is the same as before. Since E is separable, by Proposition 7.1 of Chapter 3, the finite-dimensional distributions of X and Y determine their distributions on DEIO, 00) (C,[O, 03)).

0 4.4 Corollary Let E be separable, and let A c B(E) x B(E) be linear and dissipative. Suppose that for some (hence all) rl > 0, @(A - A ) =) 9 ( A ) , and that there exists M c W E ) such that M is separating and M c 9?(1 - A ) for every rl > 0. Then for each p E 9 ( E ) any two solutions of the martingale problem for (A, p ) with sample paths in DEIO, 00) have the same distribution on DEIO, 00).

4.5 Remark Note that the significance of this result, in contrast with Theorem 4.1, is that we do not require 9 ( A ) to be separating. See Problem 22

0 for an example in which 9 ( A ) is not separating.

Proof. If X and Y are solutions of the martingale problem for (A , p ) with sample paths in DEIO, a), and if h E M, then by (4.3,

(4.28)

= E[-e-%( Y ( t ) ) dc]

for every rE > 0. Since M is separating, the identity

(4.29) r e - "E[h(X(t))] dr = r e - a f E [ h ( Y( t ) ) ] dr

holds for all h E B(E) (think of j; e-"E[h(X(r))] dr z h dv,,,). By the uniqueness of the Laplace transform, for almost every c 2 0,

(4.30) ~Ch(X(t))I = WJ(Y(t)) l . and if h is continuous, the right continuity of X and Y imply (4.30) holds for all c 2 0. This in turn implies (4.9) and the uniqueness follows from Corollary 4.3. 0

The following theorem shows that the measurability condition in Theorem 4.24~) typically holds.

4.6 Theorem Let (E, r ) be complete and separable, and let A c c ( E ) x RE). Suppose there exists a countable subset A, c A such that A is contained in the bpclosure of A, (for example, suppose A c L x L where L is a separable subspace of C(E)). Suppose that the DJO, 00) martingale problem for A is well-posed. Then, denoting the solution for (A, 6,) by P,, PJB) is Borel measurable in x for each B E 9,.

Proof. By Theorems 5.6 and 1.7 of Chapter 3, (9(DE[0, do)), p), where p is the Prohorov metric. is complete and separable. By the separability of E and Proposition 4.2 of Chapter 3, there is a countable set M c C(E) such that M is bp-dense in B(E).

Let H be the collection of functions on Da[O, 00) of the form

(4.31) rt = (ma+,)) --f(X(t"M - ~ + ' B ( x o ) ds) k- fi 1 4(X(hM I"

where X is the coordinate process, (A g) e A,, k,, ..., h, E M, 0 s rl < cz < * * . < and tk E 69. Note that since A, and M are countable, H is countable, and sincefand the it, are continuous, P E &P(Ds[O, 00)) is a solution of the martingale problem for A if and only if

(4.32) q d P = O , q~ H. s Let A, c 9(DE[0 , a)) be the collection of all such solutions. Then dA = (7, ~ ,,{P: J q d P = 0}, and d, is a Borel set since H is countable and { P : I q dP = 0) is a Borel set. (Note that if q E C(DBIO, a)), then F,,(P) 3 q dP is continuous, hence Borel measurable, and the collection of ti E

B(D,[O, 00)) for which F, is Borel measurable is bp-closed.) Let G: 9(D,[O, mi)--+ 9 ( E ) be given by G(P) = PX(O)-'. Note that G is

continuous. The fact that the martingale problem for A is well-posed implies that the restriction of G to A, is one-to-one and onto. But a one-to-one Borel measurable mapping of a Borel subset of a complete, separable metric space onto a Borel subset of a complete, separable metric space has a Borel measurable inverse (see Appendix lo), that is, letting P,, denote the solution of the martingale problem for (A, p), the mapping of 9 ( E ) into 9 ( D E [ 0 , 00)) given by p-+ f,, is Borel measurable and it follows that the mapping of E into

0 SyD,[O, 00)) given by x 3 P , zs P , is also Borel measurable.

Theorem 4.2 is the basic tool for proving uniqueness for solutions of a martingale problem. The problem, of course, is to verify (4.9). One approach to doing this which, despite its strange, ad hoc appearance, has found widespread applicability involves the notion of duality.

Let ( E , , rl) and ( E 2 , r 2 ) be separable metric spaces. Let A, c H E , ) x RE,),

p2 E B(E,). Then the martingale problems for ( A l , p,) and ( A 2 , p 2 ) are dual A2 c WE,) x B(Ez), f t~ M(Ei x E d , E M(E,) , P E M E , ) , P I E WEIX and

4. THE MARTINGALE PROBLEM: UNIQUENESS, THE MARKOV PROPERTY, A N D DUALITY 189

with respect to (f. a, p) if for each solution X of the martingale problem for ( A , , p l ) and each solution Y for ( A , , p,), &Ia(X(s))lds < 00 as.,

I8( Y(s) ) I ds < 4, a.s.,

(4.33)

and

for every t 2 0. Note that if X and Y are defined on the same sample space and are independent, then (4.35) can be written

(4.36)

4.7 Proposition Let (El, r l ) be complete and separable and let E , be separable. Let A , c W E , ) x B ( E , ) , A , c B(E, ) x B(E,), SE M(EI x E2) , and p E

M(E,). Let A t 9 ( E , ) contain PX(r)-' for all solutions X of the martingale problem for A , with PX(O)-' having compact support, and all f 2 0. Suppose that (Al, p) and (A , , 6,) are dual with respect to (f. 0, p) (i.e., a = 0 in (4.35)) for every p E R E , ) with compact support and every y E E , , and that {f(*, y): y E E,} is separating on A. Iffor every y E E , there exists a solution of the martingale problem for ( A z , 6J then for each p E B ( E , ) uniqueness holds for the martingale problem for ( A p).

4.8 Remark (3 The restriction to p with compact support in the hypotheses is important since we are not assuming boundedness for j'and fl. Com- pleteness is needed only so that arbitrary p E 9((El) can be approximated by p with compact support.

(b) The proposition transforms the uniqueness problem for A l into an existence problem for A , . Existence problems, of course, are typically simpler to handle. 0

190 GENERATORS AND MAIKOV PROCESSES

Proof. Let V, be a solution of the martingale problem for ( A 2 , d,,). If p E

*El) has compact support and X and are solutions of the martingale problem for (Al , p), then

= E c / ( m , Y)I.

Since {$(a, y) : y E €,} is separating on 4, (4.9) holds for X and 2. Now let p E 9 ( E l ) be arbitrary. If X and x are solutions of the martingale

problem for (A, , p) and K is compact with dK) > 0, then X conditioned on {X(O) E K } and 2 conditioned on (z(0) E K} are solutions of the martingale problem for (A, , p(* n K ) / p ( K ) ) . Consequently,

(4.38) f {x( t ) E r 1 x(o) E K} = p{1p(t) B rj 2(0) E K},

uniqueness. CI

r E WE,).

Since K is arbitrary and p is tight, (4.9) follows, and Theorem 4.2 gives the

The next step is to give conditions under which (4.35) holds. For the moment proceeding heuristically, suppose X and Y are independent El- and E,-valued processes, g, h E M(E, x E2),

(4.39)

is an {9f)-martingale for every y E El, and

(4.40)

is an {.F:}-martingale for every x E E,. Then

f (X(s), Y(t - s)) exp du + l-'/3( Y(u)) du}] ds

which is zero if

(4.42)

(Compare this calculation with (2.15) of Chapter 1.) Integrating gives (4.36).

dx, u) + a(xlf(x, u) = Yx, Y ) + PWJk Y).

1. THE MARTINGALE PROBLEM: UNIQUENESS, THE MARKOV ROf€RTV, AND OUAUTY 191

4.9 Example To see that there is some possibility of the above working, suppose E , = (- “3, a), E , L= {O, I , 2, ...}, A , f ( x ) =j”(x) .- xf‘(x), and A , f ( y ) = y(y - l)(J(y - 2) -S(y)). Of course A ( corresponds to an Ornstein- Uhlenbeck process and A, to a jump process that jumps down by two until it absorbs in 0 or 1. Let {(x, y) = xy. Let X be a solution of the martingale problem for A ,. Then

(4.43) X(r)’ - l ( Y ( y - I)X(S)~- - yX(s))’) ds

is a martingale provided the appropriate expectations exist; they will if the distribution of X(0) has compact support. Let g(x, y ) = y(y - I ) X Y - ~ - yxy and a(%) = 0. Then g(x, y ) = A , / ( x , y) + (y’ - 2y)xy, and we have (4.42) if we set B(y) = y2 - 2y. Then, assuming the calculation in (4.41) is justified (and it is in this case), we have

(4.44) Y2(u) - 2Y(u)) du

and the moments of X ( t ) are determined. In general. of course, this is not enough to determine the distribution of X ( t ) . However, in this case, (4.44) can be used to estimate the growth rate of the moments and the distribution is in fact determined. (See (4.21) of Chapter 3.)

Note that (4.44) suggests another use for duality. If Y(0) = y i s odd, then Y absorbs at 1 and

(4.45) 11 lim € [ X ( t y ] = lim E X(O)“(” exp ( Y2(u) - 2Y(u)) du 1-w 1 - m [ K

= 0,

since the integrand in the exponent is - 1 after absorption. Note that in order to justify this limit one needs to check that

€[exp { l ‘ ( Y ’ ( u ) - 2Y(u)) du < 00, 11 where T, = inf (I: Y ( c ) = I ) . Similarly, if Y(0) = y is even, then Y absorbs at 0 and setting = inf I t : Y(c) = O),

(4.46) lim € [ X ( t y J = E[ exp {p Y ’(u) - 2 Y(u)) du 1-m

This identity can be used to determine the moments of the limiting distribu- a tion (which is Gaussian). See Problem 23.

The next lemma gives the first step in justifying the calculation in (4.41).

192 GENERATORS AND MMKOV POCESSEI

4.10 Lemma Suppose/(s, t ) on LO, a) x LO, a) is absolutely continuous in s for each fixed t and absolutely continuous in t for each fixed s, and, setting

(4.47) [ [,fis. t ) / d s dt < o3 i = 1,2, T > 0.

Then for almost every t 2 0,

(4.48)

(.f, 9 fa) = VJ; suppose

f ( t , 0) - f ( o , f ) = lUl (s , t - s) -f&, t - s)) ds.

Proof.

(4.49) I r f(fl(s, t - s) -f,(s, t - s)) ds dr 0 0

= [ lf,(r - S, s) ds dt - S r I’/.(s, r - s) ds dt

= $’ I r f l ( t - s, s) dr ds - ~ ‘ & ( s , t - s) dt ds

= [ ( A T - s, s) -f@, 4) ds - (fh T - s) -S(s, 0)) ds

0 0

lr = $;,(s, 0) -m s)) ds.

Differentiating with respect to T gives the desired result. 0

The following theorem gives conditions under which the calculation in (4.41) is valid.

4. THE MARTINGALE PROBLEM: UNIQUENESS, THE MARKOV IROPERTV, AND DUALITY 193

and

(4.5 1) [ Ia (X(u) ) ldu + [ l / ) ( Y o ) l d u 5 c,.

l Suppose that

(4.52)

is an { +SF:)-martingale for each y, and

S ( X ( 0 , Y) - ff(X(s)* Y) ds

(4.53) S(x, W)) - p ( x , Us)) ds

is an {*#:}-martingale for each x. (The integrals in (4.52) and (4.53) are assumed to exist.) Then for almost every c z 0,

4.12 Remark Note that (4.54) can frequently be extended to more-general a

a and /I by approximating by bounded a and p.

Proof, Since (4.52) is a martingale and Y is independent of X, for h 2 0.

(4.55)

We use (4.50) and (4.51) to ensure the integrability of the random variables above.

Note that for r , s + h $ T, the absolute values of the second and fourth terms are bounded by

(4.56) +h ~[r,]e~r.

Set

(4.57) F(s, r ) =

A similar identity holds for F(s, t) - F(s, 0) and (4.54) rollows from Lemma 4.10. 0

4. N ruanwxE PROBLEM: UNIQUENESS, ME MMKOV mocERTy, AND DUALITV 19s

4.13 Corollary If, in addition to the conditions of Theorem 4.11, dx. Y) + a ( x ) / ( x , Y) = h(x, Y) + P(v)f (x. Y). then for all t 2 0,

(4.60)

Proof. By (4.54), (4.60) holds for almost every r and extends to all t since a F( t , 0) and F(0 , I ) are continuous (see (4.55)).

The estimates in (4.50) may be difiicult to obtain, and i t may be simpler to work first with the processes stopped at exit times from certain sets and then to take limits through expanding sequences of sets to obtain the desired result.

4.14 Corollary Let {PI} and (9,) be independent filtrations. Let X be {9,)-progressive and let T be an {9,}-stopping time. Let Y be {'3,}-progressive and let t~ be a {'3,}-stopping time. Suppose that (4.50) and (4.51) hold with X and Y replaced by X(. A 7 ) and Y ( . A a) and that

is a ('3,)-martingale for each x. (The integrals in (4.61) and (4.62) are assumed to exist.) Then for almost every I 2 0,

1% CMUTOIS AND MMYOV PRO-

proof. Note that (4.61), for example, can be rewritten

(4.64)

The proof of (4.63) is then essentially the same as the proof of (4.54). 0

4.15 Corollary Under the conditions of Corollary 4.14, if dx, Y) + a(x)/(x, Y) = h(x, Y) + P(y)J(x, Y), then for all t 2 0,

4.16 Remark As T , 6-t 00 in (4.65), the integrand on the right goes to zero. The dificulty in practice is to justify the interchange of limits and expectations.

0

5. THE MARTINGALE PROBLEM: EUSTENCE

In this section we are concerned with the existence of solutions of a martingale problem, in particular with the existence of solutions that are Markov or strong Markov. As a part of this discussion, we also examine the structure of the set A,, of all solutions of the DEIO, 00) martingale problem for a given A, considered as a subset of @DEIO, 00)). One of the simplest ways of obtaining solutions is as weak limits of solutions of approximating martingale problems, as indicated by the following lemma.

5.1 Suppose that for each (I, 8) E A, there exist (j, , g,) E A, such that

(5.1)

Lemma Let A c c ( E ) x c ( E ) and let A, c B(E) x E(E), n = 1,2, ... .

lim II J, -f II I 'Ql n-m

0, lim II 8, - g II = 0.

If for each n, X, is a solution of the martingale problem for A, with sample paths in &[O, a), and if X , * X , then X is a solution of the martingale prablem for A.

5. THE MARTINGALE PROBLEM: EXtSTFNCE 197 -

5.2 Remark Suppose that (E, r ) is complete and separable and that 9 ( A ) contains an algebra that separates points and vanishes nowhere (and hence is dense in C(E) in the topology of uniform convergence on compact sets). Then {X,) is relatively compact if (5.1) holds for each (J g ) E A and if for every E , T > 0, there exists a compact Ke. c E such that

(5.2) for inf P{ X,(t) E K 8 , 0 5 I 5 7') 2 I - E. n

See Theorems 9.1 and 9.4 of Chapter 3. 0

Proof. Let 0 5 t, 5 I < s, t , , f , s 6 D ( X ) = { u : P{X(u) = X ( u - )} = 11, and h, E c((E), i = 1, . . . , k. Then for ( J g) E A and (fn, 8.) E A, satisfying @.I) ,

Pn(X,(s)) -jJXn(f)l BAXAU)) du /t,(Xn(t,)) n-m -i' 1

= 0.

By Lemma 7.7 of Chapter 3 and the right continuity of X, the equality holds for all 0 I r, 5 t < s, and hence X is a solution of the martingale problem for A. 0

We now give conditions under which we can approximate A as in Lemma 5.1.

5.3 Lemma Let E be compact and let A be a dissipative linear operator on C(E) such that O(A) is dense in C(E) and ( I , 0) E A. Then there exists a sequence { T,} of positive contraction operators on B(E) given by transition functions such that

(5.4) lim n(T, - f ) J = A$ J E 9 ( A ) . n-rm

Proof. Note that

(5.5) Af 3 n(n - A ) - ' f ( x )

defines a bounded linear functional on g ( n - A ) for each n 2 I and x E E. Since A1 = 1 and I Af I 5 II f 11, forJ2 0,

(5.6) I I f I l -N'=NIIfII -1)s l l / l l p and hence Afr 0. Consequently, A is a positive linear functional on 9 ( n - A ) with II A 11 = 1. By the Hahn-Banach theorem A extends to a positive linear

1% cmuroIs AND MARUOV ~ocrms

functional of norm 1 on all of C(E) and hence by the Riesz representation theorem there exists a measure p E P(E) (not necessarily unique) such that

(5.7)

Consequently, the set

(5.8) M: = {p E 9(&): n(n - A)-'f(x) = I/. for all /E 3 ( n - is nonempty for each n 2 I and x E E.

If Iimk*- xk = x and pk E M:,* then by the compactness of 9 ( E ) (Theorem 2.2 of Chapter 3), there exists a subsequence of {pk} that converges in the Prohorov metric to a measure pa E 9 ( E ) . Since for all f E 9 ( n - A),

(5.9) J j d p - = tim J f dpk = lim n(n - A)-y(xk) = n(n - A)-y(X)*

pm E M: and the conditions of the measurable selection theorem (Appendix 10) hold for the mapping x-+ M:. Consequently, there exist cc[: E M: such that the mapping x-+ pz is a measurable function from E into 9(&). It follows that pr(x, r) = c(:(r) is a transition function, and hence

k-m k-m

(5.10) T f ( 4 = J f W " ( X , dY)

is a positive contraction operator on B(E). It remains to verify (5.4). For/€ Ea(A),

(5.1 1)

and hence

(5.12)

Since 9 ( A ) is dense in C(E) it follows that (5.12) holds for all f E C(E). There- fore forJE 9 ( A ) ,

(5.13)

since Af E C(E).

lim n(T, - I ) f = lim KAf- AJ r-. m r - m

If E is locally compact and separable, then we can apply Lemma 5.1 to obtain existence of solutions of the martingale problem for a large class of operators.

5. TME MAITINGALE PROIILEM: EXISTENCE 199

5.4 Theorem Let E be locally compact and separable, and let A be a linear operator on &5). Suppose 9 ( A ) is dense in t ( E ) and A satisfies the positive maximum principle (Lea, conditions (a) and (b) of Theorem 2.2 are satisfied). Define the linear operator A' on C(E') by

(5.14) ( A ~ S ) ( E = A((f - l (A))hd, AAf(A) = 0,

for a l l f E C(E') such that ( J - f ( A ) ) ( , E !@A). Then for each v E P(EA), there exists a solution of the martingale problem for (AA. v ) with sample paths in DKaCO, CQ).

5.5 Remark If A* satisfies the conditions of Theorem 3.8 (with l? = Ed) and v(E) = I , then the above solution of the martingale problem for (A', v ) will have sample paths in DEIO, 00). In particular, this will be the case if E is

U

Proof, Note that 9 ( A ) = C(EA) and that J(xo) = sup,, J ( y ) 2 0 impliesJ(x,)

then AbJ(xo) = 0 by definition.) Since A A l = A0 = 0, A' satisfies the conditions of Lemma 5.3 and there exists a sequence of transition functions p,(x, r) on E' x A?(EA)such that

compact and (I, 0) E A.

7

-J(A) = supr J ( y ) -f(A) 2 0 SO AY(x0) = A(S-J(A)Xx,) 5 0. (If XO = A

c

(5.15)

satisfies

(5.16)

For every v c @(E') the martingale problem for (A,,, v ) has a solution (a Markov jump process) and hence by Lemma 5. I and Remark 5.2 there exists a

0 solution of the martingale problem for ( A , v).

We now consider the question of the existence of a Markov process that solves a given martingale problem.

Throughout the remainder of this section, X denotes the coordinate process on DECOY a), Y a collection of nonnegative, bounded, Bore1 measurable functions on DEIO, 00) containing all nonnegative constants, and

(5.17) 9, = P',"v6({t s s): s sr, T E Y).

Note that all T E F are (F,}-stopping times.

Assume rv # 0 for each v and for/E E(E) define Let r c 9 ( D s [ 0 , 00)) and for each v E 9 ( E ) let rv = { P E I-: P X ( 0 ) - * = v } .

(5.18) p e r . LJo J

200 GENERATORS AND M O V PROCESSES

The following lemma gives an important relationship between y and the martingale problem.

5.6 lemma SupposeJI g E B(E) and

(5.19)

is an {.W,)-martingale for all P E rv. Then

for all P E rv.

Proof. This is immediate from Lemma 3.2. 0

We are interested in the following possible conditions on r and 9.

5.7 Conditions

C,: For P E r, T E 3, p = PX(r)'I, and P' E r,,, there exists (2 E 9(DE[0, a) x [O, 00)) with marginal Q E r such that

(5.2 1) E*CXdX(. A rtx r t ) X C ( X h + -))I

= 5 EPCXS(X(* A 71, T) I X(7) = xlE"Cxdx(*)) I X(0) = X M W

for all BEY^ x a[O, 00) and C E YE, where (X, 4) denotes the coordinate random variable on DEIO, a) x [O, to). (Note there can be at most one such (2.) C2: For P E r, T E J, and H L 0 SP,-measurable with 0 < EP[KJ < a, the measure Q E 4yD,[O, m)) defined by

(5.22)

is in r. c3 : r is con vex. C,: For each h ts c ( E ) such that h 2 0, there is a u E WE) such that ?(I-,, 41) = u dv for all u E q E ) . Cs: rv is compact for all Y E B(E).

We also use a stronger version of C, and a condition that is implied by C2.

C;: For v, p,, p 2 E @(E) such that v = ap , + (1 - a)fi2 for some u E (0, 1) and for P E rV, there exist Q , E r,,, and Q1 E r,,, such that P = aQI + (1 - a)Q2.

5. TM MARTINGALE ruomw: EXISTENCE 201

ck: Uy. rv is compact for all compact v c B(E).

5.8 Lemma Condition C, implies C; .

Proof. Then setting H I = h,(X(O)), i = 1, 2,

Let h , = dp,/dv and h, = dp2/dv, and note that ah, + ( I - a)h, = I .

(5.23)

= E P [ H i xc(X)]

is in r#,, and P = aQ, + (1 - a)Q2. 0

Condition C ; is important because of its relationship to C4. To see this we make use of the following lemma.

5.9 Lemma Let E be separable. Let cp: P(E)-*[O, c] for some c > 0. Suppose cp satisfies

(5.24) cP(ap, + (1 - a h ) = W4Pl) + ( 1 - a)rph)

for a E (0, I ) and p,, p 2 , k 9 Y E ) and that cp is upper semicontinuous in the sense that v,, v E P(E), v, * v implies

(5.25) T;;f; q ( v n ) I; CP(V). n-m

Then there exists u E &E) such that

(5.26) rp(v) = u dv, v E .9(E). I Proof. By (5.25), u(x) E ~(6,) is upper semicontinuous and hence measurable ({x: u(x) < a ) is open). Let E;, i = I , 2, .... be disjoint with diameter less than l/n and E = u, E;; let x; E E; satisfy u(x;) 2 sup,,,: dx) - l/n. Fix Y and define u, E B(E) by

(5.27)

and v, E 9 ( E ) by

(5.28)

Then bp-lim,,, u, = u and v, - w . Consequently,

(5.29) I u dv = !!it Iu, dv = !!y u dv, = lim ~ ( v , ) s q ( v ) . I n-w

202 GENERATORS AND U K O V PROC€S!WS

To obtain the inequality in the other direction (and hence (5.26)) let &B) = v(B n E;)/V(E;) when v(E;) > 0 and u,(x) = 1 cp(p;)xE;(x). Note that

u,(x) s u(x) by (5.25), and hence

(5.30)

0

5.10 Lemma Let (E, r) be complete and separable. Suppose conditions Cz and C3 hold. Then for h E E(E) with h 2 0,

Proof. Condition C; implies the right side of (5.31) is greater than or equal to the left while C3 implies the reverse inequality. If C, holds, then for v,, v E WE), v, 3 v, we have rv u un rv. compact. Consequently. every sequence Pa E has a subsequence that converges weakly to some P e ru. Since, for h E C(E), j$e-'h(X(t)) dc is continuous on &[O, ao), it follows that

(5.32)

C, now follows by Lemma 5.9. 0

Let A, be the collection of all pairs (J 8) E B(E) x E(E) for which (5.19) is an {.F,)- martingale for all P E r. Our goal is to produce, under conditions C,-Cs, an extension A of A , satisfying the conditions of Theorem 4.1 such that for each Y E tP(Q there exists in rv a solution (necessarily unique) of the martingale problem for (A, v). The solution will then be a Markov process by Theorem 4.2. Of course typically one begins with an operator A, and seeks a set of solutions r rather than the reverse. Therefore, to motivate our consideration of C,-C, we first prove the following theorem.

5.11 Theorem Let (E, r) be complete and separable.

(a) Let A c B(E) x B(E), let r = AA (recall that AA is the collection of all solutions of the D,[O, 00) martingale problem for A), and let J be the collection of nonnegative constants. Suppose rv # 0 for all v E q E ) . Then

(b) Let A c c ( E ) x C(E), and - let r = AA and F = {t: {t < c} E 9: for all c 2 0, T bounded). Suppose 9 ( A ) contains an algebra that separates points and vanishes nowhere, and suppose for each compact K c E, E > 0,

CI-C, hold.

5. THE MARTINGALE PROOLEM: EXISTENCE 203

and T > 0 there exists a compact K' c E such that

(5.33) P { X ( r ) E K' for all t < T, X(0) E K )

2 ( I - E)P{ X(0) E K ) for all P E r. Then CI-C, and C; hold.

In addition to the assumptions of part (b), suppose the &[O, a) martingale problem for A is well-posed. Then the solutions are Markov processes corresponding to a semigroup that maps c(€) into C(E).

(c)

5.12 Remark Part (b) is the result of primary interest. Before proving the theorem we give a lemma that may be useful in verifying condition (5.33). Of course if E is compact, (5.33) is immediate, and if E is locally compact with A c C(E) x c((E), one can replace E by its one-point compactification EA and

0 consider the corresponding martingale problem in D,,,[O, 00).

5.13 Lemma Let (E, r ) be complete, and let A c C ( E ) x WE). Suppose for each compact K c E and q > 0 there exists a sequence of compact K, c E, K c K,, and ( f,, 9,) E A such that for F, = {z: inf,. K m dx, z) I; q } ,

(5.34) Pn. ly inf L(v) - SUP f J Y ) 0, Y ~ K y 6 E - F .

(5.35)

and

(5.36)

Then for each compact K c E, E > 0, and T > 0, there exists a compact K' c E such that

(5.37)

for all P E A,.

P { X ( t ) E K' for all t < T , X(0) E K } 2 ( I - E)P(X(O) E K } ,

5.14 Example Let E = W'and A = {(J G f ) : f E C;(W')} where

(5.38)

and the a,, and h, are measurable functions satisfying Ia,,(x)I s M(I + 1xI2) and Ih,(x)l s M(1 -t 1x1) for some M > 0. For compact K c B(0, k ) = {z E R': I z I < k ) and q > 0, let K, = B(0, k + n ) and let f, E C?(R') satisfy

L(x) = I + log ( I + (k + )t + a)') - log ( I + I x 1')

for 1x1 5 k + n + q and 0 $ L ( x ) s I for 1x1 > k + n + q. The calculations are left to the reader. D

201 GENERATORS AND W U O V ?ROC€SS€S

proof. Given T > 0, a compact K c E, and q > 0, let F, be as hypothesized, and define T,, = 0 if X(0) $ K and T,, = inf { r : X( t ) $ Fa} otherwise. Then for P G J . 4 ,

and hence

which gives

(5.41) P { X ( t ) E Fn for all r s T, X(0) E K )

= P{X(O) E K } - P ( 0 < T, I; T }

2 PIX(O) E K ) (1 - 8,: ( T sup g;(y) + Ilfnll- inf .W)). YcFn Y ~ K

From (5.41). (5.35). and (5.36), it follows that for each m > 0 there exists a compact t?, c E such that

(5.42) P{X(t) E R,!,', for all t s T , X(O) E K}

2 P{X(O) 6 K}(1 - 82-3.

Hence taking K' to be the closure of om l?,!,'"', we have (5.37). 0

In order to be able to verify C1 we need the following technical lemmas.

5.15 Lemma Let (E, r), (S, , pl), and (S,, p2) be complete, separable metric spaces, let P , E P(S,) and P, E 9 ( S J and suppose that XI: S, 4 E and X , : S 2 4 E are Bore1 measurable and that p E B(E) satisfies p =I P , X ; ' =

5. THE MARTINGALE PROWEM: EXISTENCE 205

P, X; I . Let {BY} c a ( E ) , m = 1, 2, . . ., be a sequence of countable partitions of & with {B;""} a refinement of (ST} and limm-m sup, diameter (B;") = 0. Define P" E 9(SI x S2) by

(5.43)

for C E W ( S , x S,). Then {P"} converges weakly to a probability measure P E 9 ( S 1 x S,) satisfying

(5.44) P(AI x A2) = E P ' [ ~ A l I XI = x J E P ' [ ~ A 2 I = x ] ~ ( d x )

for A l E @S,) and A , E a@,). In particular P(A, x S,) = P , ( A , ) and P ( S , x A 2 ) = P2(A2). More generally, if

(5.45)

E B(&), k = I , 2, then

EP[Z, 2 2 J = E P ' [ Z I I x, = x ] E ' * [ Z 2 I x, = x]p(dx) . I Proof. For k = I , 2, let A, E g ( S k ) . Note that EP'[~,,, I X , = x ] is the unique (pas.) #(E)-measurable function satisfying

(5.46) l E " [ X A & I 'k = xl/c(dx) = E P k L X A k X d x k ) l

for all B E A?(E). By the martingale convergence theorem (Problem 26 of Chapter 2),

(5.47)

p a s . and in E(p). Consequently,

(5.48) lim P"(A, x A , ) = P(A, m-m

(P (A , x A2) given by (5.44)), and since at most one P E 9(Sl x S,) can satisfy (5.44). it suflices to show that {P"} is tight (cf. Lemma 4.3 of Chapter 3). Let E > 0, and let K I and K , be compact subsets ofS, and S, such that Pk(Kk) 2 I - &'. Then, Since fk(Kk) 3

(5.49)

&I, I x k = X]p(dX),

p ( X : E [ x K , I X k = x ] 5 1 - E } 5 6

and

(5.50) P(K1 x KJ 2 ( I - &)'(I - 2 ~ ) .

Tightness for (P"} now follows easily from (5.48). 0

206 CPlUATORS AND W Y O V PROCESSES

5.16 lemma Let (E, t ) be complete and separable, and let A c B(E) x B(E). Suppose for each v E 9 ( E ) there exists a solution of the martingale problem for (A, v ) with sample paths in D,[O, a). Let 2 be a process with sample paths in DEIO, 00) and let r be a [O, 001-valued random variable. Suppose, for ( J 8) E A, that

(5.51)

is a martingale with respect to 9, = @(SAT), SAT: s s 2). If r is discrete or if B(A) t C(E), then there exists a solution Y of the martingale problem for A with sample paths in Ds[O, ao) and a [O, a]-valued random variable q such that (Y( * A q), q) has the same distribution as (Z(* A f ) , T).

Proof. Let P 6 9(Dr[0 , 00) x 10, 003) denote the distribution of (2, t) and p E f l E ) the distribution of Z(r) (fix xo E E and set Z(t) = xo on { r = 00)). Let P, E S(D,[O, 00)) be a solution of the martingale problem for (A, p). By Lemma 5.15 there exists Q c 9(DE[0 , 00) x [O, 001 x DBIO, ao)) such that, for E E Y E x O[O, 0 0 3 and C E YE,

(5.52) Q(B x C) = EP' [~B(X , q) I X(q) = x]E"[xdX) I X(0) = x]p(dx) s = I ECXe(Z T) I Z(T) - x 1 ~ ' " x d ~ ) I X ( 0 ) = xIp(dJ4

where ( X , q) denotes the coordinate random variable on DEIO, 00) x 10, a]. Let ( X , , q, X,) denote the coordinate random variable on Q = DEIO, a) x [0, 001 x DEIO, m) and define

(5.53)

Note that on (Q, @Q), Q), Y( A q) = X,( A q) has the same distribution as Z ( . AT). It remains to show that Y is a solution of the martingale problem for A.

With reference to (3.4). let (/t g) E A, hk E q E ) , t , < t2 < t , < * * - < r,,, , and define

5. THE MARTINGALE PROBLEM: EXISTENCE 207

We must show E*[R] = 0. Note that

(5.55)

S; R , + R , .

Since R , is zero unless tn < q, we have

(5.57) for q < oo

V m =

By the right continuity of X2 and the continuity off, as m-, 00 RT converges a.s. to R , . Noting that R! = 0 unless q,,, < f,, we have

(5.59) EQCRT3 - c ~ Q c R ~ x , q , = , , m l l I<mi,+l

X Ep' [ X f q m = ~ / m ) n h k ( x ( f h ) ) I k 4 I/n

= 0,

since P2 is a solution of the martingale problem for (A, p). Letting m-+ 00, we see that EQ[R2] = 0.

If 9 ( A ) $ C(E) but q is discrete, then EQ[R2] = 0 by the same argument as in (5.59). 0

Proof of Theorem 5.11 (a). (C,) Let P E r, f E Y, p = PX(r ) - I , and P E r,, . In the construction of Q in the proof of Lemma 5.16 take P , ( B ) = P { ( X , T ) E B } for B E Y E x a[O, a ] and P2 = P'. Then the desired 0 is the distribution of (Y, q ) defined by (5.53) on (n, @a), Q). Note that Lemma 5.16 applies under either the conditions of part (a) or of part (b).

(C2) Let P c r, r E F, and H 2 0 and 4,-measurable with 0 < E P [ H ] < 00. Define Q by (5.22). Then for (5 8) E A, hk E B(E), and t , < t , < - . * < t ,+ , ,

= 0,

since H is 9,-measurable and P E A,, . (Under the assumptions of part (b), the continuity ofjallows the application of the optional sampling theorem.) (C3) The set of P E 9(DE[0, 00)) for which (3.4) holds is clearly convex.

Proof of Theorem 5.11 (b). To complete the proof of part (b) we need only verify C; (which implies C5), since C, will then follow from Lemma 5.10.

5. THE MARTINGALE PUOWLW: EXISTENCE 209

(C!) Let V c 9 ( E ) be compact. Then for 0 < -E c 1 and T > 0, by Theorem 2.2 of Chapter 3, there exist compact K c E such that w(K) 2 1 - c/2 for all v E V and (by (5 .33)) compact K c , c E such that

(5.61) P { X ( t ) E K # , r for all t < T }

2 P ( X ( r ) E Ke, r for all t < T , X ( 0 ) E K )

for all P E uveY rv . The following lemma completes the proof of C5 and hence of part (b).

5.17 Lemma Let (E, r ) be complete and separable. For t, T > 0, let K r , r c E be compact and de&Kz f = {x E D,[O, GO): x ( t ) E K e , r for all t < T ) . If A c C(€) x B(E) and 9 ( A ) contains an algebra that separates points and vanishes nowhere, then

(5.62) ( P E A , : P ( K : , ) z I - - E forall E. T > 0 }

is relatively compact. If, in addition, A c C(E) x C(€), then (5.62) is compact.

Proof. The relative compactness follows from Theorems 9.1 and 9.4 of Chapter 3. If A c c(€) x c((E), then compactness follows from Lemma 5.1 with A, =Afar all n. Note that K?r is closed, and hence P , =- P implies P(K: r) 2 P A C r). 0

Proof of Theorem 5.11 (c). Let P , denote the solution of the D,[O,'oo) martingale problem for ( A , ax). By C; and uniqueness, P, is weakly continuous as a function of x, and hence by Theorem 4.2 the solutions are Markov and correspond to a semigroup { T(r)}. By Theorem 3.12, P , { X ( t ) = X ( t - ) f = I

0 for all t and the weak continuity of P , implies T(t ) : (5(E)-+ C(€).

We now give a partial converse to Lemma 5.6, which demonstrates the importance of condition C4.

5.18 Lemma Let c .9(DE[0, 00)) and 5 satisfy C,. Suppose u, h E B(E) and

(5.63)

for all P E and w E 9 ( E ) . Then for each P E r, rt

is an {#',}-martingale.

210 GENERATORS AND MMKOV rRoassEs

proof. Let P E I', r 2 0, B E 9, with P(B) > 0, and v(C) = P{X(r) E Cl 8) for all C E 5o(E). Then with Q given by (5.22) for H = xB,

(5.65) e' 5. I'e-''h(X(u)) du dP

= l e - ' h ( X ( t + s)) ds dP

= P ( B ) E Q [ r e - ' h ( X ( s ) ) ds]

I: P(B) I u(x) dv - Lu(X(r)) dP.

= P(BhC,, h)

Hence

(5.66)

= e-'u(X(r)) + e-'h(X(s)) ds. l Since (5.66) is clearly a martingale, the lemma follows from Lemma 3.2. 0

5.19 Theorem Let (E, r) be complete and separable. Let J be a collection of nonnegative, bounded Bore1 measurable functions on DEIO, a) containing all nonnegative constants, and let (9,) be given by (5.17). Let I' c SyD,[O, 00))

and suppose r, # 0 for all v E @E). Let A. be the sct of (J Q) Q B(E) x B(E) such that

(5.67)

is an (9,)-martingale for all P E I-. Assuming C,-C,, the following hold:

' (a) There exists a linear dissipative operator A 3 A . such that &(I - A ) = S(€) (hence by Lemma 2.3 of Chapter 1, 5i?(A - A) = W E ) for all 1 > 0) and 9 ( A ) is bpdense in B(E).

(b) Either r, is (L singleton for all Y or there exists more than one such extension of Ao.

(c) For each v E P ( E ) there exists P, E I-", which is the unique (hence Markovian) solution of the martingale problem for (A, v), and if P, is the unique solution for (A, ax), then P, is a measurable function of x and P, = P, v(dx).

(d) Every solution P of the martingale problem for A satisfies

(5.68)

for all C c Y E and T E 9.

P { X O + .) E C I 4 , ) = PX(,)(C)

Proof. Let JI, f2, . . . E C(E) be nonnegative and suppose the span of { f k f is bp-dense in B(E) (such a sequence always exists by Proposition 4.2 of Chapter 3). Let I-c0) = I- and rlo) = r,. Define

for all v E P ( E ) and set re’) = u rik). Since rt0) is nonempty and compact and P + EP[j$ e-yl(X(t)) dt] is a continuous function from 9((oE[o, 00)) to R, it follows that I-‘:’ is nonempty and compact and similarly that rtk’ is nonempty and compact for each k. The key to our proof is the following induction lemma.

5.20 lemma Fix k L 0, and let rck) be as above. If Pk) and 9 satisfy CI-C,, then Jcr+I) and Y satisfy Cl-C5. (We denote these conditions by qk) and qk+ as they apply to rck’ and Fkt I ) . )

Proof. implies

Let p E 9 ( E ) and P E r?+’). For B E g ( E ) with 0 < p(B) < I , Cf)

where pr(C) = p(B n C)/AB) and p2(C) = p(B‘ A C)/p(B) for all C E g ( E ) , and the inequality holds term by term. But C;’), Cik’ and Lemma 5.10 imply equality holds in (5.70), so by C t ) there exists a u k + I E E(E) such that

Hence

We now verify C\k+ ‘)-C?+ I).

212 GENERATORS AND W K O V PROCESSES

(C:+'J) For P E I-;*+lJ c I E Y, f l = f X ( r ) - I , and P' c r p c rf), there exists 0 with marginal Q c ckJ such that (5.21) holds. We must show Q E l?,"+'J. Let p*(dx) = (E'[e-'I X ( r ) = x] /EP[e- ' ] )p(dx) . Then, using (5.21). (5.72), and CtJ,

By CTJ there exists P" E r,l) such that

Hence equality must hold in (5.74). so Q E r\*+'). Let f E Ptl) and r E 9, and let p* be as above. Then for

B E .Ft with 0 < f ( B ) < I, Cf' and the fact that equality holds in (5.74) imply

(@:+'))

5. THE MARTINGALE PROELEM: EXISTENCE 213

1 + E p [ xa, e - ' [ e - y k + t ) ) dr

5 E P [ ~ e e - ' l y ( r $ ) , h t + E'[x, e - ' ~ r ( r $ ! j i + ,I, where p:(C) = EP[~ee-'~c(X(r))]/EP[~se-'] and &(C) = EP[XB. e-' xc(X(r))]/EP[x, e-'1 for all C E g ( E ) . As before CyJ, CyJ, and Lemma 5.10 imply equality in (5.79, and since the inequality is term by term we must have

(5.76)

which implies

(5.77)

Now let H 2 0 be .F,-measurable with 0 < E P [ H ] < m. Then Q, given by (5.22), is in Pk', and setting v = QX(0)- I , (5 .77 ) implies

(5.78)

214 CMUUTORS AND MARYOV PROOSSES

for p l , p2 E @I?) and 0 < a < 1, which in turn implies the convexity of rfk + I ),

(@:+'I Let u k + L be as above. By Cf), for h E C(&) with h 2 0 and E > 0, there exists u, E B(E) such that

(5.80) y(r:), x + + ~ h ) = U, d ~ , u E ~P(E) .

We claim that for each x E E

(5.82) u du = y(rtk+I', h). s First observe

(5.83) I U c dv 2 / U k + 1 dv + &Y(rik+'), h), V f g((E),

and in particular u, 2 ftk+ and

(5.84)

For each E > 0, let P, E rtk) satisfy

lim E - ~ ( U , ( X ) - u k + I ( X ) ) 2 y(rlt+l), h). T O

Clearly limc,o u, dv = y(r!kJsh+ and by the continuity of jz e-Ayk+l(X(c)) dt, all limit points of {f'} as e-*O are in rik+I! Con- sequently,

(5.86) lim / E - ' ( v . - L ( k + i ) du r - 0

In particular,

(5.87) lim & - I ( U , ( X ) - U ~ + ~ ( X ) ) s y ( q ! + I ) , h).

Thercfore (5.81) holds and since 0 5 e-'(u, - uk+ I ) S Ilhll, (5.82) follows by the dominated convergence theorem.

- r - 0

(@,f+')) This was verified above. 0

5. THE MARTINGALE PROMEM: EXISTENCE 215

Let rfm) = pk) and for each k 2 0 let Proof oi Theorem 5.19 continued. uk+, be as above. Then, by Lemma 5.18, for each k 2 0,

(5.88)

is an {9,)-martingale for all P E r"+l). hence for all P E Pm'. Note that rim) # 0 for all v E B(E) since I-f* I ) c F:), rLk) # 0 for all k, and I-!!' is compact. Let A be the collection of ( J , g) E E(E) x E(E) such that (5.67) is an {$,)-martingale for all P E P). Then A 3 A. and since ( u k + , , uk+, - f k + ,) E

A for k = I, 2, . . . , L@(/ - A ) contains the linear span of (h) and hence equals B(E). By Proposition 3.5, A is dissipative, hence by Lemma 2.3 of Chapter I @,I - A ) = B(E) for all A > 0. Lemma 3.2 implies

uk+ l(x(t)) - (uk+ I(x(s)) -h+ l(x(s))) ds L

(5.89)

for every P E I-::). Therefore i f f € C(E), then

(5.90)

and it follows that .$@(A) is bp-dense in E(E), which gives part (a). Since A satisfies the conditions of Theorem 4.1, the martingale problem for

( A , v), v E 9(E), has at most one solution, and rim' # 0 implies it has exactly one solution.

If I-" is not a singleton for somc v E S ( E ) , then there exist P, P' E rV and k > 0 such that

(Otherwise I-, = I-:'' for all k and I-" = rtrn).) Therefore reptacingh by l l fkl l

- X for all k in the above procedure would produce a different sequence

Let ko be the smallest k for which there exist wo and P, P' E Two such that (5.91) holds. Then r::) # r\t). in fact r!:) n fl") = 0, for k > k o . Consequently r::) # r:;) and the extension of A. corresponding to llhll - A differs from A.

Let P , denote the unique probability measure in rb;'. The semigroup { T(r)} corresponding to A (defined on g ( A ) ) can be represented by

(5.93) T(r)S(x) = € p ~ C f ( ~ ~ ~ ) ) l .

216 GENERATORS AND W O V PROCESSES 1 - -

Since T(t): B(A)-, 9 ( A ) and @A) is bp-dense in WE), { T(t)) can be extended to a semigroup on all of B(E) satisfying (5.93). Consequently,

(5.94) f i t , X, r) = ~~~cXr(x(t))i

is a transition tunction, and by Proposition 1.2 function of x for all B f YE. For each v E iP(E),

(5.95) P, s 1 P, v(dx)

P J B ) is a Bore1 measurable

is a solution of the martingale problem for (A, v ) and hence is the unique element of rLm). This completes the proof of part (c).

Since rob satisfies C2 for all k, Far satisfies C,. For P E Po), T e Y, and B E 9, with P(B) > 0, uniqueness implies

(5.96)

where p ( D ) = EPIXaXo(X(r))]/P(B),for all D E AV(E). Since B is arbitrary in gFI,

(5.68) follows. 0

6, THE MARTINGALE PROBLEM: LOCAl fZATlON

Let A c B(E) x B(E), let U be an open subset of E, and Ict X be a process with initial distribution v E P(E) and sample paths in &[o, 00). Define the {Sf)-stopping time

(6.1) T -- inf { t 2 0: X(r ) # U or X ( t - ) $ U}.

Then X is a solution of the stopped martingale problem for ( A , v, V) if X( ) = X( * A r) as. and

(6.2) J ( X ( t ) ) - fi cdx(s)) ffs

is an {$:)-martingale for all (5 g) E A. (Note that the stopped martingale problem requires sample paths in DEIO, a).)

6.1 Theorem Let (E, r ) be complete and separable, and let A c C(E) x B(E). If tne DEIO, 00) martingale problem for A is well-posed, then for each v E 9 ( E ) and open V c E there exists a unique solution of the stopped martingale problem for (A, v, V) .

Proof. Let X be the solution of the DE[O, 00) martingale problem for (A , v), define T by (fil), and define % ( a ) = X ( . h r ) . Then 2 is a solution of the

6. THE MARTINGALE PROBLEM: LOCALIUTION 217

stopped martingale problem for (A , v, U) by the optional sampling theorem (Theorem 2.13 of Chapter 2).

For uniqueness, fix v and V and let X be a solution of the stopped martingale problem for (A , u, V). By Lemma 5.16 there exists a solution Y of the DEIO, co) martingale problem for (A, v ) and a nonnegative random variable q such that X (= X(. A t)) has the same distribution as Y ( . A v). Note that in this case the q constructed in the proof of Lemma 5.16 is inf {I 2 0: Y(r) 9 V or Y ( f - ) 6 V}, and since the distribution of Y is uniquely determined, i t follows that the distribution of Y(. A 9) (and hence of X) is uniquely determined. 0

Our primary interest in this section is in possible converses for the above theorem. That is, we are interested in conditions under which existence and, more importantly, uniqueness of solutions of the stopped martingale problem imply existence and uniqueness for the (unstopped) D,[O, 00) martingale problem. Recall that uniqueness for the D,[O, co) martingale problem is typically equivalent to the general uniqueness question (cf. Theorem 3.6) but not necessarily (Problem 21).

6.2 Theorem Let & be separable, and let A c ( f (E) x B(E). Suppose that for each u E P(E)'there exists a solution of the DE[O, 00) martingale problem for (A , v). If there exist open subsets U,, k = I , 2, . .., with E = ur= I V k such that for each u E P(E) and k = 1, 2, ... the solution of the stopped martingale problem for ( A , u, U,) is unique, then for each u E 9 Y E ) the solution of the DE[O, 00) martingale problem for (A, u ) is unique.

Proof. Let V, , V,, . . . be a sequence of open subsets of E such that for each i there exists a k with = V k and that for each k there exist infinitely many i with V; = Uk. Fix u E 9 ( E ) , and let X be a solution of the martingale problem for (A , v ) . Let T,, = 0 and for i 2 I

(6.3) r i = i n f ( t r t , _ , : X ( r ) $ V, or X(t- )$ 6).

We note that lim,-m T , = GO. (See Problem 27.) For/€ C(E)and I. > 0,

(6.4) E[S-a-"/(X(t)) dc]

218 CWMTORS AND W K O V ?ROCESSES

For i 2 1 such that P { r , - < 00) > 0, define

for B E B(E), and

(6.7)

for C E Y E . Let Y; be the coordinate process on (DEIO, a), 9,. P,). Then 8 is a solu-

tion of the stopped martingale problem for (A, F , , v), and hence, given p i , its distribution Pi is uniquely determined. Set

(6.8) y, = inf { t : x ( t ) $ % or I#-) $ q), Then for i 2 1 with P { r , < 00) > 0

lw, < mi xdX(ri))l E'e-kt-1 - h i X X I ; , - , < g, j

i y , < mj xd YhJ)Iai - 1

Pi + ,(B) = Qi

(6.9)

E P I [ ~ - A Y I x z

9

ai

where

(6.10) af = ECe-"'Xcc<mJ = ECe-Ar"-iXtc-l<mi - 4i Xisr<miI

= E'ICe - Ayi~17, 1 1 ~ , - I

1 n EpACe-AyA~~yl :<mJ. k = I

Consequently, P , , . . . , P, determine pi+, , which in turn determines P I + . Since p1 = v , it follows that the P, are the same for all solutions of the martingale problem for (A, v ) with sample paths in DJO, 00). But the right side of (6.4) can be written

(6.1 1)

so that (6.4) is the same for all solutions of the DEIO, 00) martingale problem for (A, v), and since 1 is arbitrary, the uniqueness of the Laplace transform implies E [ f ( X ( r ) ) ] is the same for all solutions. (Note E['(X(r))] is right continuous as a function of I.) Since f E C%!((E) is arbitrary, the one-dimensional distributions are determined and uniqueness follows by Corollary 4.3. 0

Note that the proof of Theorem 6.2 uses the uniqueness of the solution of the stopped martingale problem for (A , p, U,) for every choice of p. The next result does not require this.

6. THE MARTINGALE PROBLEM: LOCAUUTlON 219

6.3 Theorem Let (E, r ) be complete and separable, and let A c C(E) x B(E). Let U , c U , c be open subsets of E. Fix v E *&), and supposc that for each k there exists a unique solution xk of the stopped martingale problem for (A . v, u k ) with sample paths in DEIO, 00). Setting

(6.12) ?k = inf ( t : xk(l) f!! u k or xk(t-) f!! Uk},

suppose that for each c > 0,

(6.13)

Then there exists a unique solution of the D,[O, cn) martingale problem for ( A , 4.

Proof. Let

(6.14) T: = inf (1: X,(r) f!! U k or X,(t-) 4 uk}.

For k < m, X,(. A T:) i s a solution of the stopped martingale problem for (A , v, uk) and hence has the same distribution as x k . It follows from (6.13) that there exists a process X, such that Xk==-Xm. (In particular, for the metric on DEIO, ao) given by (5.2) of Chapter 3, the Prohorov distance between the distributions of xk and X, is less than €[e-"^"'].) In fact, for any bounded measurable function G on D,[O, 00) and any T > 0,

(6.15) IE[G(Xk('A r))1 - E[G(X,('AT))]I 5 211GIIP{Tk 5 r). Let T: be defined as in (6.14). Since the distribution of X,,,(.A r i ) does not

depend on m 2 k, i t follows that the distribution of XJ.A T:) is the same as that of XI,. Hence

(6.16) / (x , ( t A rL,)) - g(X,(.# ds

i s an {Sfm)-martingale for each k. Since Iimkdm r t = a0 a.s. (P{r$ 5 t ) = P(fk s c)), we see that X, is a solution ot the: martingale problem for ( A , v ) with sample paths in DEIO, a). If X is a solution of the D,[O, 00) martingale problem for ( A , v ) and

(6.17) yk = inf ( I : X(c) # uk or X ( t - ) $ uk},

then X(.Ay,) has the same distribution as X,, and hence X has the same distribution as X,. 0

ihr:

6.4 Corollary Let ( E , r ) be complete and separable. Let A,, k = I , 2, ..., A c C(E) x B ( E ) and suppose there exist open subsets U , c U z c . . . with u k uk = ESWh that

(6.1 8) { ( J X U . 8 ) : (J 8) = {(J x U l g ) : ( J g ) 6 A ) .

220 GENERATORS AND MARKOV P R O M

(In the single-valued case this is just B(Ak) = 9 ( A ) and A, f Iu, = Af Iu, for all / c 9 ( A ) . ) If for each k, the DBIO, 00) martingale problem for A k is well-posed, and if for each v E @E), the sequence of solutions {xk} of the DEIO, 00)

martingale problems for (A,, v), k = 1, 2, . . ., is relatively compact, then the DEIO, oo) martingale problem for A is well-posed.

Proof. Any solution of the stopped martingale problem for (A, v, u k ) is a solution for the stopped martingale problem for ( A k , v, uk) and hence is

8, = Xk('A Tk) is the solution of the stopped martingale problem and (6.13) follows from the relative compactness of {xk}. Theorem 6.3 then gives the desired result. 0

UniqUe by Theorem 6.1. Set Tk inf { t : xk(t) f! Or xk(t -) f! Uk).*Then

The following lemma is useful in obtaining the monotone sequence (uk} in Theorem 6.3.

6.5 Lemma Let E be locally compact and separable, and let U, , U2 be open subsets of E with compact closure. Let A c C(E) x B(E), and suppose B(A) separates points. If for each v E @(E) and k = 1, 2, there exists a solution of the stopped martingale problem for (A, v, uk), then for each v E 9(E) there exists a solution of the stopped martingale problem for (A, v, U , u Uz).

Proof. Let V, = U, for k odd and = U 2 for k even, and fix x, c E. Lemma 5.15 can be used to construct a process X and stopping times ti such that r0 = 0, t, is given by (6.3). and X ( t ) = X& - T,), t, 5 f < T , + , , where X, is a solution of the stopped martingale problem for (A, pi , t;), po = v, and pr(r) = P{X(T, ) E r, t, < 00) + 6,,(r)P{T, = oo}. Let T - = lim,+m T , and note that

(6.19) S ( W A f-1) - nrmdX(S)) d3

is an (9;f-martingale for every ($, 8) E A. Either t , = 00, 7, = T - < 00 for i 2 i, (some I,) or T , < T~ < oo for all i 2 0.

In the second case ti+ , = T , implies X(t,) $ I( and hence X(T,) $ U, u U2. In the third case, the fact that (6.19) is a martingale implies limf+tm-. / ( X ( f ) ) exists for f~ 9 ( A ) . and the compactness of mc and the fact that B(A) is separating imply fimferm.- X(r) exists. But either X( t , ) or X(T,- $ V,, and hence Iirn,+*-- X(r) $ U , u U2. Consequently, 7 = inf ( t : X ( t ) $ U, u U z or X ( t - ) $ U , u Ul} s T - , and X ( . A r ) is a solution of the stopped martingale

0 problem for (A, v, L', u U2).

6.6 Theorem Let E be compact, A c C ( E ) x WE), and suppose that 9 ( A ) separates points. Suppose that for each x E E there exists an open set U c E with x E U such that for each v E B(E) there exists a solution of the stopped

7. THE MARTINGALE mommi: GENERALIZATIONS 221

martingale problem for (A, v, U). Then for each v E P(E) there exists a solution of the D,[O, a) martingale problem for ( A , v).

Proof. This is an immediate consequence of Lemma 6.5. 0

7. THE MARTINGALE PROBLEM: GENERALIZATIONS

A. The Timedependent Martingale Problem

It is natural to consider processes whose parameters vary in time. With this in mind let A c B(E) x B(E x [O, 00)). Then a measurable E-valued process X is a solution of the martingale problem for A if, for each (J g) E A,

is an { *S:}- martingale. As before, X is a solution of the martingale problem for A with respect to {9,}, where Y, 3 *.F',", if (7.1) is a {g,}-martingale for each ( J g) E A. Most of the basic results concerning martingale problems can be extended to the time-dependent case by considering the space-time process Xo(r) = (X(r), 1).

7.1 Theorem Let A c B(E) x B(E x [O , 00)) and define A' c E(E x [ O , 00))

x E(E x LO, wO))by

(7.2)

Then X is a solution of the martingale problem for A with respect to (9,) if and only if the space-time process Xo is a solution of the martingale problem for A' with respect to (Y,}.

A0 = {(SY. YY + / y ' ) : ( J ; 9 ) E A, Y E C C O , a))}.

Proof. If X is a solution of the martingale problem for A with respect to {Y,}, then for ( J g) E A and y ts Cl[O, a),

(7.3)

is a {9,}-martingale by the argument used to prove Lemma 3.4. The converse 0 follows by considering y = 1 on [O, TI , T > 0.

Suppose (7.1) can be written

(7.4)

222 CENmioRs AND MAIKOV ruocxsss

where { A(r): I 2 0) is a family of bounded operators. In this case in considering the space-time process the boundedness of the operators is lost. Fortu- nately, the bounded case is easy to treat directly. Consider generators of the form

(7.5)

where A E M([O, 00) x E) is nonnegative and bounded in x for each fixed t , p(r, x, a ) E P(E) for every (t, x) E [O, 00) x E, and p(., ., r) E B([O, ao) x E) for every I- E 1(E). We can obtain a (time inhomogeneous) transition function for a Markov process as a solution of the equation

P(S, t , X, r) = W) + 4 ~ , XI MU, t , Y, r) (7.6) l I

- &, t, X, r)lP(u, X, d ~ ) du.

7.2 Lemma Suppose there exists a measurable function y on [O, m) such that A(t, x) s v(t ) for all t and x and that for each 7’ > 0

(7.7)

Then (7.6) has a unique solution.

Proof. We first obtain uniqueness. Suppose P and set

(7.8)

Since M is nonincreasing in s, it is measurable in s and

are solutions of (7.6) and

M ( S , t ) = SUP SUP I P(U, t , X, r) - f lu , t , X, r) 1. I. r J S U S I

(7.9) M(s, t ) s y(u)ZM(u, t ) du. 6’ A slight modification of Gronwall’s inequality (Appendix 5) implies

Existence is obtained by iteration. To see that the solution is a transition M(s, 1 ) = 0.

function it is simplest to first transform the equation to

(7.10) P(s, t , X, r) = 6,(r) exp { - i l A b , x) du}

7. THE MARTINGALE PROELEM: GENERALIZATIONS 223

(To see that (7.10) is equivalent to (7.6), differentiate both sides with respect to s.) Fix t and set Po(s, t , x, r) = aX(r) and

(7. I 1) P" "(s, t , x, J) = 6,( J) exp {-[Xu. x) du{

Note that for each n, P"(s, t , x, a ) E q E ) , and for each r E W(E), P"(., ., .. r) is a Bore1 measurable function. Let

~ , ( s , t) = SUP SUP I P + l(u, t , x . r) - PW, t , x, r)i. X . T ~ S U S I

Then

(7.14)

From (7.14) we conclude that {P"(s, t , x, r)} is a Cauchy sequence whose limit must satisfy (7.6). 0

7.3 Theorem Let 1 and p be as above, and define

I f A satisfies the conditions of Lemma 7.2, then the martingale problem for A is well-posed.

Proof. The proof is left as Problem 29. 0

224 GENERATORS AND W K O V PRocEssEs

6. The Local-Martingale Problem

If we relax the condition that the functions in A be bounded, the natural requirement to place on X is that

(7.16) .f(x(t)) - l 8 ( x ( s ) ) ds

be a local martingale. (In particular if we drop the boundedncss assumption, (7.16) need not be in 2.) Consequently, for A c M ( E ) x M ( E ) we say that a measurable €-valued process X is a solution of the local-martingale problem for A if for each (I; g) E A

(7.17) l l g ( X ( s ) ) l ds c OL) 8,s.

and (7.16) is an {*ff}-local martingale. For example, let

(7.18) A = { (A W"):/E C'(R)}.

Then the unique solution of the local-martingale problem for (A, v), v E qR) , is just Brownian motion with mean zero, variance parameter 1, and initial distribution Y. It8's formula (Theorem 2.9 of Chapter 5) ensures that (7.16) is a local martingale.

Let A , c B(E) x B(E). Let p be nonnegative and measurable (but not necessarily bounded) and set

(7.19) Az = { ( A fig): (A 8) e A , } .

If Y is a solution of the martingale problem for A , and t satisfies

(7.20)

then X = Y(T(.)) is a solution of the local-martingale Chapter 6.)

problem for A, . (See

Many of the results in the previous sections extend easily to local- martingale problems.

C. The Martingale Problem Corresponding to a Given Semigroup

Let { T(t)) be a semigroup of operators defined on a closed subspace L c B(E). Then X is a solution of the martingale problem for {T( t ) } if, for each u > 0 and/E L,

(7.21) VJ - t ) / ( W ) )

is a martingale on the time interval [O, u] with respect to (9:). Of course if L is separating, then X is a Markov process corresponding to { T(t)} .

8. CONWRGENCE TMOREMS 225

8, CONVERGENCE THEOREMS

In Theorem 2.5 we related the weak convergence of a sequence of Feller processes to the convergence of the corresponding semigroups. In this section we give conditions for more general sequences of processes to converge to Markov processes and allow the limiting Markov process to be determined either by its semigroup or as a solution of a martingale problem.

If a sequence of processes {Xn} approximates a Markov process, i t is reasonable to expect the processes to be approximately Markovian in some sense. One way of expressing this would be to require

lim EC I E C / ( X n ( t + s)) ISPI - V s ) S ( X n ( t ) ) I I = 0

where (T(s ) ) is the semigroup corresponding to the limiting process. The following lemma shows that a condition weaker than (8.1) is in fact suflicient.

n - m (8.1)

8.1 Lemma Let (E, r) be complete and separable. Let ( X , ) be a sequence of processes with values in E, and let { T(t)} be a semigroup on a closed subspace L c C(E). Let M c c(E) be separating, and suppose either L is separating and { X,(t)} is relatively compact for each t 2 0, or L is convergence determining. I f X is a Markov process corresponding to { T(t ) } , X,(O) - X ( 0 ) , and

= 0 1 n-m [ i - I

k

Iim E (S(Xn(t + s)) - T(s)S(Xn(t))) n gXXn(t,)) (8.2)

f o r a l l k r O , O < t l < t 1 < . . . < r , 5 t < t + s , S ~ L , a n d 8 , ,..., q k ~ M , t h e n the finite-dimensional distributions of X , converge weakly to those of X.

Proof. (8.2) implies

(8.3)

Let f E L and t > 0. Then, since X,,(O) =t. X ( 0 ) and T(t)f is continuous,

lim ECS(xn(t))l = lim EC Vt).f(Xn(O))l n-m n-m

= ~C7-~~)S(X(O))I = ECS(X(O)I,

and hence X,,(t)*X(t), using Lemma 4.3 of Chapter 3 under the first conditions on L. Fix m > O and suppose (Xn(rl), ..., Xn(tJ)-(X(fl), ..., X(t,)) for all 0 5 f , < t , < . . . .e I,,, . Then (8.2) again implics

(8.4) m

lim E [ S ( x n ( t m + 1)) n gdxn(ti))] 11-03 I = I

226 GENERATORS AND MMYOV PROCESSES

for all 0 s r , < I , < < t , , , , f E L, andg,, . . . ,gm E M. Since relative compactness of (X,(t)} , t 2 0, implies relative compactness of {(x,(t,). ..., X,,(r,,,+ ,}I, we may apply Lemma 4.3 and Proposition 4.6, both from Chapter 3, to conclude (xn(t11, . -., Xl(lm+ 1)) r* (WJ, . . , x ( r m + k)). 0

The convergence in (8.1) (or (8.2)) can be viewed as a type of semigroup convergence (cf. Chapter 2, Section 7). For n = 1, 2, . . . let {Yr:} be a complete filtration, and let 9, be the space of real-valued {g:]-progressive processes C: satisfyin8

for each T > 0. Let 2, be the collection of pairs (4, 9) 6 9, x 9, such that

is a {g;}-martingale. Note that if X, is a {Yr:}-progressivt solution of the martingale problem for A, c B(E) x B(E) with respect to {Y:j, then ( f o X, , g o X,)€~, foreach(f ,g)EA, .

8.2 Theorem Let (E, r ) be complete and separable. Let A c c ( E ) x c ( E ) bc linear, and suppose the closure of A n e r a t e s a strongly continuous contraction semigroup (T(r)j on L = 9 ( A ) . Suppose X, , n = I, 2, ..., is a {$:}-progressive E-valued process, X is a Markov process corresponding to { T(t)), and X,,(O) X(0). Let M c c ( E ) be separating and suppose either L is separating and { X i t ) } is relatively compact for each r L 0, or L is convergence determining. Then the following are equivalent:

(a) The finite-dimensional distributions of X, converge weakly to those of x.

(b) For each (J g) E A,

for all k 2 0,O s t l e t , c= -= tk s t < t + s, and R ,,..., h, 6 M.

0. CONVERGENCE TMORLMS 227

(c) For each (J g) E A and T > 0, there exist (Cn, cp,) E d,such that

8.3 Remark (a) Note that (8.10) and (8.1 I ) are implied by

(8.1 2) lim EC I C n ( t ) -f(xAt)) I3 = lim EI: I VAt) - S(Xn(t))I I = 0. n-. m n - Q

If this stronger condition holds, then (8.1) will hold. Frequently a good choice for (tn, cp,) will be (b)

(8.13)

and

(8.14) cpn(t) = G ' EC/(Xm(t + 6,)) -/(xn(f)) I T I for some positive sequence ( e m } tending to zero. See Proposition 7.5 of Chapter 2.

(c) Conditions (8.9) and (8.1 I ) can be relaxed. In fact i t is sufficient to have

for all k 2 0, 0 s r , < t Z c below.

(d) 9 ( A ) c C ( E ) provided/€ L and h E M imply/. h E Land

c tk s t 5 T, and h , , ..., hk E M. See (8.23)

For the implication ( c - a ) we may drop the assumption that

(8.16) lim ~C/(X.(O))l = ECS(X(O))l, f E L.

need not be continuous. 0

n-m

Note that (8.16) may be stronger than convergence in distribution since/

Pruof. (a 3 b) This is immediate.

and (8.10) follows from (8.7) and the dominated convergence theorem. (c *a) Let (1; g) E A, and let ((,, 9,) be defined by (8.17) and (8.18). We

claim that {(en, cp,)} satisfies (8.8H8.11) with T replaced by 00. As above, it is enough to consider (8.10). Let T > 0 and let (I, @,,) E 2, satisfy (8.8)- (8.11) for all k 2 0,O s t , < ... < t, 5 t s T, and h, , .. ., hk E M. Then

(8.20)

is a {g;}-martingaIe (by the same argument as used in the proof of Lemma 3.2), and for 0 s t S T,

e-'tm(t) + ~ - Y C ~ ( U ) - $m(lO) du sb

Jc Let k z 0.0 s I , < . . - < r,, s t s T, and h,, . . . , h, E M. Then

r k 1

8. CONVERGENCE THEOREMS 229

The first term on the right of (8.22) can be written, by (8.2 I), as

k 1

1 k

- E [ Q t + S) - + S) 1971) ds n hAX,,(t,)) . I = I

As n-+ 00 the second term on the right of (8.22) goes to zero by (8.10), as do the second and fourth terms in (8.23). (Note that the conditioning may be dropped, and the dominated convergence theorem justifies the interchange of limits and integration in the fourth term. Observe that (8.15) can be used here in place of (8.9) and (8.1 I) . ) Consequently,

Since T is arbitrary the limit is in fact zero. Let 9'; be the Banach space of real-valued (Y:l-progressive processes t

with norm lltll = sup, E[ 1t(t)1]. Define n,,: L - r 9," by n , , j ( t ) - f ( X , , ( t ) ) , and for t,, E U,", n = 1,2, ... , a n d l c L, define LIM C, =/ifsup,, IIt,II < 00

and

for all k 2 0.0 4 t , < . + . < t , 5 t,and h , ,.... hk E M. Let (.T,(s)} denote the semigroup of conditioned shifts on 9':. Clearly

LIM t,, = 0 implies LIM Y,,(s)t,, = 0 for all s 2 0, and LIM satisfies the conditions of Theorem 6.9 of Chapter 1. For each (S, g ) E A, we have shown there exist (em, cp,) E d,, such that LIM (,, =S and LIM cp,, = 9. Conse- quently, Theorem 6.9 of Chapter t implies

for all /E L and s 2 0. But (8.26) is just (8.2), and hence Lemma 8.1 implies (a). 0

8.4 Corollary Suppose in Theorem 8.2 that X,, 3: 9. 0 Y. and (g:} = {SF}, where Y. is a progressive Markov process in a metric space En corresponding to a measurable contraction semigroup { q(r)) with full generator A,,, and qn: Em-+ E is Borel measurable. Then (a), (b), and (c) are equivalent to the following:

(d) For each (1; g) E A and T > 0, there exist (f,,, g,,) E A,, such that {(t,,, cp,,)} = {(& 0 Y. , g,, 0 Y.)} satisfies (8.8H8.11) for all k 2 0, 0 5 t , < < tk S I 5 T,and h , , ..., hk E M.

Proof. (d * c ) It only needs to be observed that (S,, g,,) E A,, implies

(c 3 d) By the Markov property, (t,,, 9,) defined by (8.17) and (8.18) is of the form (S, 0 Y., g,, 0 V,) for some (J , , g,,) E J,,, and (d) follows by (8.24).

0

( L o Y . * g a o Y , ) E ~ . *

8.5 Corollary Suppose in Theorem 8.2 that X,, = q,,(Y,([an.])) and (3:) = {SFk,,), where { x ( k ) , k = 0, I , 2, .. .) is a Markov chain in a metric space E,, with transition function c(,,(x, r), q,,: E N + E is Borel measurable, and a,,- 00

as it--, ao. Define T,: B(E,,)-+ B(E,,) by

(8.27)

and let A,, = a,,(T, - I). Then (a), (b), and (c) are.equivalent to the following:

(el For each (1; 8) E A and T > 0, there exist f, c WE,,) such that for Bn = An /n 3 ((4. 9 Vn)) all k 2 0.0 5 t , <

{(LA Y.(Can*I)), en( W a n *I)))} satisfies (8.8W8. I 1 ) for < tk < r s T,and h ,,..., hk E M.

and

(8.30)

and note that

(8.31) gn = A n f n = f e + ring - nn1:

For f 3: k/an (k E Z+), (8.30) gives

(8.32)

JnC U C a n tl))

Since (a,/(l + = e - * uniformly in s 2 0, the result now 0 follows as in the proof of (b + c).

8.6 Corollary Suppose in Theorem 8.2 that the X, and X have sample paths in D,[O, m), and there is an algebra C, c L that separates points. Suppose either that the compact containment condition (7.9) of Chapter 3 holds for { X , } or that C, strongly separates points. I f I((,. cp,)} in condition (c) can be selected so that

(8.33) lim S U P I <,(I) -J(xA~))I] = 0 n - m 1 e Q n 1 0 . T )

and

(8.34) SUP ECIl~nlJp, TI < 03 for Some P E: (I, 001,

then X, - X. n

Proof. This follows from Theorems 9.4, 9.1, 7.8 and Corollary 9.2, all of 0 Chapter 3. Note that D in that chapter's Theorem 9.4 contains 9 ( A ) .

8.7 Corollary Suppose in Theorem 8.2 that the X, and X have sample paths in DEIO, oo), and that there is an algebra C, c L that separates points. Suppose either that the compact containment condition ((7.9) of Chapter 3) holds for (X,,) or that C, strongly separates points. If X, has the form in Corollary 8.4 and n, f - f o q,, then either of the following conditions implies x, * x.

(f) For each (f, g) E A and T > 0, there exist (h, g,,) E A,, and G, c En such that { x(r) E G,, 0 s r 5; T} are events satisfying

(8.35) l i r n P { x ( r ) e G , , O s r s T}=1, 1-m

SUP, ItfnII < oo,and (8.36) Iim SUP I n, f ( ~ ) -.L(Y) I = lim SUP I n, &) - gncV) I 0.

a-m rsG. ,-.In ycG.

@I For eachfe L and T > 0, there exist G, c En such that (8.35) holds and

(8.37) lim sup I 7 J r ) q / ( y ) - n, T(t)fQ) I = 0, 0 5 r s T. n-rm ~ I G .

8.8 Remark (1) If G, = En, then (8.35) is immediate and (f) and (g) are equivalent by Theorem 6.1 of Chapter 1.

(b) If the Y. are continuous and the G, are compact, then the assumption that sup, 11/;11 < 00 can be dropped. In general it is needed. For example, with Em = E = (0, 1,2, ...} let

(8.38)

(Clearly, if X, has generator A, with X,(O) = 0, then X, * X where X e 0.) Let

A, f ( 4 = n - '4df(n) - f (O)) .

(8.39) Af(k) = d O k ( f ( l ) - f (O)) .

Set G, = (0, 1, 2, ..., n - I } and

(8 .4 ) fXk) = f ( k ) + adn(S(1) -.f(O))*

Then (8.41) A n f a ( k ) = d O k ( f ( 1 ) -f(o) + n-'(f(n) -.f(O)))*

and hence (8.42) lim sup I h ( k ) - f ( k ) ) = lim sup I A,/;(k) - Af(k)] = 0

a-m k s G . r -m LcG.

suggesting (but not implying!) that X,,*z, where R has generator A. Of 0 course sup,, 11/,11 = 00.

I [e-' I W n A f - gHy) - n, WXS - gKY) I dt

+ 2e-V.f- 911.

Using (8.37). the dominated convergence theorem, and the arbitrariness of T , a sequence G, can be constructed satisfying (8.35) (for each 7' > 0) and (8.36). Consequently (8) implies (f).

To see that (f) implies X, * X, fix (f, g) E A and T > 0. Assuming (L, g,) E A, and G, c En are as in (f), define

Note that (8.35) and (8.36) imply P ( r , < T ) = 0. Set

(8.46)

(8.15). 0

tn(t) = fn( Yn(t A Tn)h Vn.(r) = Sn( Yn(l))Xltn > 1 ) .

Then (, and cp, satisfy (8.33) and (8.34) with p = 2 as well as (8.8). (8.10), and

8.9 Corollary Suppose in Theorem 8.2 that the X, and X have sample paths in D,[O, 00) and that there is an algebra C, c L that separates points. Suppose either that the compact containment condition ((7.9) of Chapter 3) holds for {X,} or that C, strongly separates points. If X, has the form in Corollary 8.5 and n , J = j o q,, then either of the following conditions implies X, * X:

(h) For each (A g) E A and T > 0, there exist f , E: E(E,) and G, c En such that ( Y.([a,, r ] ) E G,, 0 s I s T } are events satisfying

(8.47) lim P { Y.([a,f] ) E G , , 0 s t 5 T ) = I , n-m

234 GENERATORS AND MARKOV PRROCESSES

(i) For eachJE L and T > 0, there exist G, c En such that (8.47) holds and

(8.49) lim sup I TP'z, f ( y ) - K,, T(t)f(y) I = 0, 0 s t s T. n-m yeG.

Proof. The proof is essentially the same as that of Corollary 8.7 using (8.30) in 0 place of (8.43), and (8.28) (appropriately stopped) in place of (8.45).

We now give an analogue of Theorem 8.2 in which the assumption that the closure of A generates a strongly continuous semigroup is relaxed to the assumption of uniqueness of solutions of the martingale problem. We must now, however, assume a priori that the sequence (Xn) is relatively compact. Note that (9:) and d,, are as in Theorem 8.2.

8.10 Theorem Let (E, r ) be complete and separable. Let A c C(E) x e ( E ) and u E B(E), and suppose that the &LO, 00) martingale problem for (A, u ) has at most one solution. Suppose X , , n = 1,2, . . . , is a {g;)-adapted process with sample paths in D,[O, a), {X"} is relatively compact, PX,(O)-' - u, and M c € ( E ) is separating. Then the following are equivalent:

(a3 There exists a solution X of the D d O , 00) martingale problem for (A, u), and X , =L X.

(b3 There exists a countable set r c [O, 00) such that for each (s, 8) E A

(8.50)

(c’) There exists a countable set r c [O, 00) such that for each ( J g) E A and T > 0, there exist (C, cp,) E -2, such that

(8.51)

(8.52)

8.11 Remark As in Theorem 8.2, (8.52) and (8.54) can be replaced by (8.15). c1

Proof. (a’ +b’) Take r = 10, 00) - ax). ( D ( X ) = { I 2 0 : P { X ( t ) = X(r-)} = l} .) By Theorem 7.8 of Chapter 3, (x,,(I,), ..., xn(tk))-(X(c,), ..., x(tk)) for all finite sets {I, , t 2 , . . . , tk} c D ( X ) , and this implies (8.50).

(b’ =+ c’) The proof is essentially the same as in Theorem 8.2. (c ‘ea‘ ) Let Y be a limit point of {Xn}. Let ( J 9) E A and T > 0,

and let {(en, cp,)) satisfy the assertions of condition (c’). Let k 2 0,

M. Since O s t , < . . . < t k I , t < t + s j T with t i . t , C + S E D ( Y ) and h , ,..., k k E

it follows that

(8.56)

f(xn(t + s)) -f(x,(t)) - dXn(u)) du n+m c”’

and hence

f ( Y ( t + s)) -f(Y(r)) - B(Y(u)) du n hAY(ti)) = 0. I”’ 1 By the right continuity of Y, (8.55) holds for all 0 I; I , < . . - < t k 5 f c t + s, and hence Y is a solution of the martingale problem for ( A . v). There-

0 fore (a’) follows by the assumption of uniqueness.

We state analogues of Corollaries 8.4-8.7 and 8.9. Their proofs are similar and are omitted.

8.12 Corollary Suppose in Theorem 8.10 that X, = q,, 0 U, and {Y:} = {Fp], where Y. is a progressive Markov process in a metric space En corresponding to a measurable contraction semigroup { T , ( f ) } with full generator A,,, and 1,: En- E is Borel measurable. Then condition (f) of Corollary 8.7 implies (a'), and (a'), (b), and (c') are equivalent to the following:

(d') There exists a countable set lr c [O, a) such that for every (f, g) e A and T > 0, there exist (f., g,) E. A,, such that {(t,, cp,)} = {(jn a x , g, o YJ) satisfies (8.51)-(8.54) for all k 2 0, 0 S t l < < t k S t s T with t i , f 6 r, and h , , . .., hk E M.

8.13 Corollary Suppose in Theorem 8.10 that X , = qa 0 Y.([u;]) and {g;} = {S~ , , ) , where { Y,(k), k = 0, 1,. . .}. is a Markov chain in a metric space En with transition function p,(x, r), q,,: E n d E is Borel measurable, and u,,+ 03 as n-, 03. Define T,: B(E,)-, @En) by (8.27) and let A, = a,(T, - I). Then condition (h) of Cbrollary 8.9 implies (a'), and (a'), (b), and (c') are equivalent to the following:

(el) There exists a countable set r c [0, 03) such that for every (1; a) E A and T > 0, there exist f, E HE,) such that for g, = A,f,, {(&, cp,)} = {(L(x([ct,,*])), g,(X,([(ccc,. 3)))) satisfies (8.51H8.54) for all k 2 0, 0 5 t l < . . . < t , ~ t t 5 w i t h t , , t I r , a n d h , , .... h,~M.(Notethatweare not claiming (en , cp,) E d, .)

8.14 Remark In the following three corollaries we do not assume a priori that { X,} is relatively compact. We do assume the compact containment condition. The assumption that C, strongly separates points used in the analo-

0 gous corollaries to Theorem 8.2 does not suffice.

8.15 Corollary Let (E, r ) be complete and separable and let {X,} be a sequence of processes with sample paths in Ds[O, a). Suppose fX,(O)-' =+ Y E

9 ( E ) and the compact containment condition ((7.9) of Chapter 3) holds. Suppose A c c ( E ) x c(€), the closure of the linear span of 9 ( R ) contains an algebra that separates points, and the Ds[O, 00) martingale problem for (A, v ) has at most one solution. If ((C,, cp,)) in condition (c') of Theorem 8.10 can be selected so that (8.33) and (8.34) hold, then (a') holds.

8.16 Corollary Instead of assuming in Corollary 8.12 that {X,} is relatively compact, suppose that {A',) satisfies the compact containment condition ((7.9) of Chapter 3) and the closure of the linear span of J ( A ) contains an algebra that separates points. Then condition (f) of Corollary 8.7 implies condition (a') of Theorem 8.10.

8.17 Corollary Instead of assuming in Corollary 8. I3 that { X , ) i s relatively compact, suppose that { X,} satisfies the compact containment condition ((7.9) of Chapter 3) and that the closure of the linear span of Q ( A ) contains an algebra that separates points. Then condition (h) of Corollary 8.9 implies condition (a‘) of Theorem 8.10.

The following proposition may be useful in verifying (8.7) or (8.50) and as a result gives an alternative convergence criterion.

8.18 Proposition Let X , , A, and 148:) be as in Theorem 8.2. Let (J g) E A. For n = 1, 2, , . . , let 0 = r i < T; < . . - be an increasing sequence of {‘J:}-stopping times with r; < co as. and limk4m r: = a3 a.s. Define

(8.58) r:(t) = min ( r i : r; > t } ,

(8.59) rn-(t) = max { r i : r; 5 I ) ,

and

m

(8.63) lim E[ 1 H,( t ) ( J = 0, r 2 0, n+m

then

for all s, r 2 0, which in turn implies, by (8.19),

(8.65) lim EC I <At) -S(xntt)) I 1 n-m

= lim EC I cpn(t) - g(X,(t))l 1 = 0, t 2 0, n - m

where (,, and q,, are given by (8.17) and (8.18).

238 GENERATORS AND N K O V PROCESSES

Proof. For any {g;}-stopping time T and any 2 such that €[ 121 3 < 00,

(8.67)

By (8.61), (8.62), and the dominated convergence theorem, I,,(t) converges to zero in t‘. The quantity in (8.64) is bounded by €(I Hn(t + s) I) + €(I H,(t) I)

0 + € ( I 1At)l) and the limit follows by (8.63).

9. STATIONARY DlSTRl6UllONS

Let A c E(E) x B(E) and suppose the martingale problem for A is well-posed. Then p E @(E) is a stationary disrriburion for A if every solution X of the martingale problem for ( A , p) is a stationary process, that is, if P{X(t + s,) E r, , X( t + sp) E r2, . .., X(r + s,) E T,) is independent of t 2 0 for all k 2 1, 0 s s, < - . < s,, and Ti , ..., r, E a(€).

The following lemma shows that to check that p is a stationary distribution it is sufficient to consider only the one-dimensional distributions.

9.1 lemma Let A c W E ) x W E ) and suppose the martingale problem for A is well-posed. Let p E 9 ( E ) and let X be a solution of the martingale problem for (A, p). Then p is a stationary distribution for A if and only if X( t ) has distribution p for all t 2 0.

9. STATIONARY DISTRIBUTIONS 239

Proof. The necessity is immediate. For sulliciency observe that X, 5 X(r + a )

is a solution of the martingale problem for ( A , p), and hence, by uniqueness, has the same finite-dimensional distributions as X . 0

9.2 Proposition Suppose A generates a strongly continuous contraction semigroup (T(r)} on a closed subspace L c B(E), L is separating , and the martingale problem for A is well-posed. I f D is a core for A and p E 9 ( E ) , then the following are equivalent:

6) cc is a stationary distribution for A. (b) T ( t ) f d p = j f d p , J E L, 1 2 0. (c) A f d p - 0 , /E D.

Proof. (a * b) If X is a solution of the martingale problem for ( A , p), then by (4.2) and (a),

(9.1) ECW/(Wo))l= EMWJ = ~u~x(o))I, /E L, t 2 0,

which is (b). (b *a)

(4.2) and (b), Let X be a solution of the martingale problem for ( A , p). By

(9.2) ~c/(.w)I = ~C~WSW(O))I = ECS(X(O))I, SE L, r 2 0.

Since L is separating, X ( c ) has distribution p for each t 2 0, and (a) follows by Lemma 9.1.

(b * c ) This is immediate from the definition of A.

(c * b) Since A is the closure of A restricted to D, we may as well take D = 9 ( A ) . Then, by (c),

(9.3) J(T(r)f-J) dp = [ L A T W d s d p

= [ A T ( s ) / d p ds = 0.

for each f E 9 ( A ) and t 2 0. Since Q(A) is dense in L, (b) follows. Cl

If { T(t)) is a semigroup on B(E) given by a transition function and condition (b) of Proposition 9.2 holds (with L = B(E)), we say that p is a srarionary distribution for { T(r)}.

An immediate question is that of the existence of a stationary distribution. Compactness of the state space is usually suflicient for existence. This observation is a special case of the following theorem.

9.3 Theorem Suppose A generates a strongly continuous contraction semigroup { T(r)} on a closed subspace L c C(E), L is separating, and the martingale problem for A is well-posed. Let X be a solution of the martingale problem for A, and for some sequence r, Q), define (1") c 9 ( E ) by

(9.4) p,(r) = t ;* p i x ( s ) E r} ds, r E . q E ) .

If p is the weak limit of a subsequence of {p,,}, then p is a stationary distribution for A.

9.4 Remark The theorem avoids the question of existence of a weak limit point for {pa}. Of course, if E is compact, then (pa} is relatively compact by

0 Theorem 2.2 of Chapter 3, and existence follows.

Proof. Since the sequence { r , ) was arbitrary, we may as well assume @,,a@. For f E L and t 2 0, T(t) f E L c C(E), so

1 mf dp = lim J W)f 4, # e r n

(9.5)

= lirn t;' p [T ( t ) / (X (s ) ) ] ds #-a

= lirn t;' T(t + s) f dv ds I-. 9)

where v = PX(0) - ' , and hence p is a stationary distribution for A by Proposi- tion 9.2. 0

We now turn to the problem of verifying the relative compactness of {p,,) in Theorem 9.3. Probably the most useful general approach involves the construction of what is called a Lyapunov function. The following lemmas provide one approach to the construction. For diffusion processes, It& formula (Theorem 2.9 of Chapter 5 ) provides a more direct approach. See also Problem 35.

9.5 Lemma Let A c IE(E) x W E ) and let cp, $ E M ( E ) . Suppose there exist {(h, & ) ) c A and a constant K such that

(9.6) O S f k J ; C p , k - 1 . 2 ,...,

(9.7)

and

(9.9) lim d x ) = $(x), x E E . k - m

If X is a solution of the martingale problem for A with E[q(X(O) ) ] < 00,

then

(9.10)

is an { +F:)-supermartingale.

9.6 Remark Note that we only require that the gk be bounded above. 0

Putting (9.13) and (9.14) together we have

and it follows that (9.10) is a supermartingale.

242 GfNERAlORS AND W O V fROtEsSES

9.7 lemma Let E be locally compact and separable but not compact, and let Ed = E u {A} be its one-point compactification. Let cp, J, E M(E) and let X be a measurable E-valued process. Suppose

(9.16)

is a supermartingale, rp ;r 0, J, $ C for some constant C, and lims-A J,(x) = - co. Then {pl: r 2 1) c P(E), defined by

(9.17)

is relatively compact.

Proof. Given m z 1, select K, compact so that

(9.18)

Then, for each r 2 1,

r PI 1

P{X(s) E K,} ds - mr,

and therefore

= PcXK-1.

Consequently, the relative compactness follows by Prohorov’s theorem (Theorem 2.2 of Chapter 3). 0

9.8 Corollary Let E be locally compact and separable. Let A c B(E) x B(E) and rp, I,9 E M(E). Suppose that cp 2 0, that I,9 s C for some constant C and limXw4 +(x) = - 00, and that for every solution X of the martingale problem

for A satisfying E[cp(X(O))] < 00, (9.16) is a supermartingale. If X is a stationary solution of the martingale problem for A and p = P X ( O ) - ' , then

(9.2 1 )

where K, = {x: $(x) 2 - m } .

Proof. Let a > 0 satisfy P{lp(X(O)) 5 a } > 0, and let Y be a solution of the martingale problem for A with P { Y E: E } = P { X E El cp(X(0)) I a} (cf. (4.12)). By (9.20)

P(cp(X(0)) s a} s fi t - * * P{ Y( s ) E K,) ds P(cp(X(0)) I; a} m

(9.22) - C + m r-m

/ 4 K m ) .

Since a is arbitrary, (9.21) follows. 0

If X is a Markov process corresponding to a semigroup T(t): C(€)-+ c ( E ) , then we can relax the conditions on the Lyapunov function cp and still show existence of a stationary distribution even though we do not get relative compactness for { p I : t 2 1).

9.9 Theorem Let E be locally compact and separable. Let { T(t)} be a semigroup on B(E) given by a transition function P(t, x. r) such that T(t): C(E)-+ C(E) for all L 2 0, and let X be a measurable Markov process corre_seonding to {T(t)}. Let cp, + E M ( E ) , cp 2 0, $ s C for some constant C, and limx+A +(x) < 0, and suppose (9.16) is a supermartingale. Then there is a stationary distribution for { T(t)} .

Proof. Select E > 0 and K compact so that supx, $(x) s - E . Then, as in the proof of Lemma 9.7,

(9.23) n . - I

for all t 2 1, where p, is given by (9.17). By Theorem 2.2 of Chapter 3, {p,} is relatively compact in iP(EA). Let v E 9 ( E A ) be a weak limit point of {p,) as t--, 00, and let vE be its restriction to €. It follows as in (9.5) that for nonnegative/€ e (~) , (9.24) I j d V , = I / d V = !!: I T ( t ) / d p r , 2 T ( f ) l d v E . I

2 u m m m m s AND MUOV PROCWES

Note that if T ( t ) f ~ &T), then we haye equality in (9.24). By (9.23), YE(€) > 0 so p = vdvAE) E WE) and

(9.25)

for all nonnegativefe t ( E ) and r 2 0. By the Dynkin class theorem (Appendix 4), (9.25) holds for all nonnegativef E s(E), in particular for indicators, so

(9.25)

for all r E a(E) and t 2 0. But both sides of (9.26) are probability measures, so O we must in fact have equality in (9.26) and hence in (9.25).

The results in Section 8 give conditions under which a sequence of processes converge in some sense pa limiting process. If {X,} is a sequence of stationary processes and X, 6 X or, more generally, the finite-dimensional distributions of X, converge weakly to those of X, then X is stationary. Given this observation, if {A,} is a sequence of generators determining Markov processes (i.e., the martingale problem for A, is well-posed) and if, for each n, H, is a stationary distribution for A,, then, under the hypotheses of one of the convergence theorems of Section 8, one would expect that the sequence {p,} would converge weakly to a stationary distribution for the limiting generator A. This need not be the case in general, but the following theorem is frequently applicable.

9.10 Theorem Let (E, r) be complete and separable. Let {q(r)}, {T(r)} be contraction semigroups corresponding to Markov processes in E, and suppose that for each n, p,, E 9 Y E ) is a stationary distribution for {%(I)}. Let L c e ( E ) be separating and T(r): L- , L for all r 2 0. Suppose that for each S E L and compact K c E,

(9.27) lim sup I x(r)/(x) - T(r)f(x) I = 0, t 2 0. 1-10 x a K

Then every weak limit point of {p,} is a stationary distribution for { T(r)}.

9.11 Remark Note that if xm+ x implies T$)f(x,)+ T( t ) f (x) for all t 2 0, then (9.27) holds. 0

9. STATIONARV DISTRIBUTIONS 245

proof. For simplicity, assume p, =$ p. Then for eachf E L, t 2 0, and compact K c E,

(9.28)

5 lim 2iI/ tIpn(Kr)* n-m

But by Prohorov's theorem (Theorem 2.2 of Chapter 3), for every E > 0 there is a compact K c E such that p,(K') < E for all n. Consequently, the left side of (9.28) is zero. 0

Theorem 9.10 can be generalized considerably.

9.12 Theorem Let (En, t,), n = I , 2, ..., and (E, t ) be complete, separable metric spaces, let A, c B(E,) x B(E,) and A c B(E) x B(E), and suppose that the martingale problems for the A, and A are well-posed. For v, E P(E,) (respectively, v B q E ) ) , let X; (respectively, X") denote a solution of the martingale problem for (A,, v,) (respectively, ( A , v)) . Let q,: &,-+ E be Bore1 measurable, and suppose that for each choice of v, E 9 ( E , ) , n = l , 2, . . ., and any subsequence of { v , , q ; ' } that converges weakly to v E P(E), the finite- dimensional distributions of the corresponding subsequence of { q , o X;} converge weakly to those of X'. If for n = 1, 2, . . . , pl is a stationary distribution for A,, then any weak limit point of { p , , q i 1 ] is a stationary distribution for A.

Proof. If a subsequence of {p,,q,-'} converges weakly to p, then the finite dimensional distributions of the corresponding subsequence of {q,, 0 X?) converge weakly to those of X". But q. 0 X:" is a stationary process so X p must also be stationary. 0

The convergence of the finite-dimensional distributions required in the above theorem can be proved using the results of Section 8. The hidden difliculty is that there is no guarantee that {p,,q;'} has any convergent sub- sequences; thus, we need conditions under which {p,,q,-'} is relatively compact. Corollary 9.8 suggests one set of conditions.

216 CENEUTOIS AND MAIKOV PROCESSES

9.13 Lemma Let En, E, A,, A, and q, be as in Theorem 9.12, and in addition assume that E is locally compact. Let q,,, I),, E M(E,) and @ E M(E). Suppose that cp,, 2 0, #, s # 0 q,, I,$ s C for some constant C, that 1imx-,, $(x) = - 00, and that for every solution X, of the martingale problem for A, with EC~n(Xn(0))l < as

(9.29)

is a supermartingale. For n = 1, 2, ..., let pn be a stationary distribution for A,. Then, for each m, n 2 1,

(9.30)

where K, = {x: $(x) 2 - m } , and hence {p,q[ '} is relatively compact.

Proof. Let X, be a solution of the martingale problem for A, with E[qn(Xn(0))] < 00. Then, as in (9.19),

(9.3 1)

LJo -I

- [ S z L ( q m - 0 0 xn(s)) ds] 9

and the estimate follows by the same argument used in the proof of Corollary 9.8. 0

Analogues of Theorem 9.12 and Lemma 9.13 hold for sequences of discrete-

We give one additional theorem that, while it is not stated in terms of parameter processes. See Problems 46 and 47.

convergence of stationary distributions, typically implies this.

9.14 Theorem Let { TJr)} , n = 1, 2, ..., and {T(t)} be strongly continuous semigroups on a closed subspace L c C(E) corresponding to transition functions P,(t, x, r), n = 1, 2, . . . , and P(t, x, r). Suppose for each compact K, c E and E > 0 there exists a compact K, c E such that

(9.32) inf inf inf P,(k x, K,) z 1 - E. X C W I n I

Suppose that for each/€ L, c,, > 0, and compact K c E ,

(9.33)

and suppose that there exists an operator n: L--+ L such that for each and compact K c E,

lim sup sup I T , ( r ) / ( x ) - T(t)f(x)l = 0, r - m X E X ( $ 1 0

L

(9.34) lim sup I T ( r ) / ( x ) - nj(x)I = 0 l - rm X E K

and

(9.35) lim sup sup I T,,(r)n/(x) - n/(x)I = 0.

Then for each/€ L and compact K c E,

(9.36) lim sup sup I G(c)/(x) - T(t )S(x) ) = 0.

n-.m I S K Osl i rn

m-.m r 6 K O S f < m

9.15 Remark Note that frequently nf(x) is a constant independent of x, that is, n f ( x ) = I f d p where p is the unique stationary distribution for { T(t)} . This result can be generalized in various ways, for example, to discrete time or to

0 semigroups on different spaces. The proofs are straightforward.

Each term on the right can be made small uniformly on compact sets by first taking to sufficiently large and then letting n-r 00. The details are left to the reader. 0

We now reconsider condition (c) of Proposition 9.2. Note that D is required to be a core for the generator of the semigroup. Consequently, if one only knows that the martingale problem for A is well-posed and not that the closure of A generates a semigroup, then Proposition 9.2 is not applicable. To see that this is more than a technical difficulty, see Problem 40. The next theorem gives conditions under which condition (c) of Proposition 9.2 implies p is a stationary distribution without requiring that A (or the closure of A ) generate a semigroup.

We need the following lemma.

248 GENERATORS AND MARKOV PIOCLSSES

9.16 Lemma Let A c C(E) x e (E) . Suppose for each v 9(€) that the martingale problem for (A, v ) has a solution with sample paths in D,JO, a). Suppose that cp is continuously differentiable and convex on G c W", that (f, 9 gr), ...) (-Am, Qm) 6 A, ( 1 1 , ..*,fm): E + G, and that (cp(S1 ,fz, ...,f,,), h) E A. Then

(9.38) h 2 V d f i ,f,z* ***,fm) * (Sl, g,, * * . * gm)*

for all r > 0. Dividing by t and letting t -+ 0 gives (9.38). 0

9.17 Theorem Let E be locally compact and separable, and let A be a linear operator on e ( E ) satisfying the positive maximum principle such that 9 ( A ) is an algebra and dense in C(E). If p E 9 ( E ) satisfies

(9.40)

then there exists a stationary solution of the martingale problem for (A, p).

I A f d p = 0, f € 9w,

Proof. Without loss of generality we may assume E is compact and (1,O) E A. If not, construct AA as in Theorem 5.4 and extend p to @Ed) by setting p ( ( ( A } ) = 0. Then A' and p satisfy the hypotheses of the theorem. If X is a stationary solution of the martingale problem for (A', p), then f ( X ( t ) E E} = p(E) = 1, and hence X has a modification taking values in E, which is then a stationary solution of the martingale problem for (A , p),

Existence of a solution of the martingale problem for (A, p) is assured by Theorem 5.4, but there is no guaranree that the solution constructed there will be stationary. For n = 1,2, ..., let

(9.41) A, = { (A n[(f - n - ' A ) - Y - j ] ) : / € @(f - n - ' A ) } .

For f e 9 ( A ) and f , = ( I -n - 'A)J1 we see that l l f , - ~ i l - + O and IIA,f, - Af II -+ 0 (in fact A, f" = Af). It follows from Theorem 9.1 of Chapter 3 that if X , is a solution of the martingale problem for A,, n = 1, 2, 3, . . . , with sample paths in Ds[O, a), then (X,} is relatively compact.

By Lemma 5.1, any limit point of {X,} is a solution of the martingale problem for A. Consequently, to complete the proof of the theorem it sufices

9. STATIONARY DISTRIIUTIONS 249

to construct, for each n L I . a stationary solution of the D,[O, 00) martingale problem for ( A " , p). Note that for / E 9 ( A m ) , f = ( I - n - ' A ) g for some g E 9 ( A ) and

(9.42)

The key step of the proof is the construction of a measure v E .P(E x E) such that

(9.43) v(r x E ) = Y ( E x r) = p(t-1, r E a(&), and

(9.44) I h ( x ) g ( y ) v ( d x x dy) = h(xHI - n - l A ) 'g(.u)p(dx)

for all h E C(E) and g E W(/ - n - I A ) . Let M t C(& x E) be the linear space of functions of the form

I (9.45)

for h, , ..., h,, f E C(E), and g1 , .,., 9,. E 9 ( 1 - n - - ' A ) , and define a linear functional A on M by

(9.46)

Since A1 = 1, the Hahn-Banach theorem and the Riesz representation theorem give the existence of a measure v E P ( E x E) such that AF = F dv for all F E M (and hence (9.43) and (9.44)) if we show that IAFI 5 IIFII. This inequality also implies that if F L 0, then llFll - AF =:

A(IIFII - F) 5 IIIIFll - Fll 5 llFll. SO AF 2 0. Letfl .JZ, ....I, E 9 ( A ) , let a, = ]ISk - nL1AhlI , and let cp be a polynomial

on Iw" that is convex on [ -aI , a,] x [ - a 2 , a2] x . . . x [ -a,, a,,,]. Since 9 ( A ) is an algebra, cp(fl , . . . ,$,) E Q(A), and by Lemma 9.1 6,

(9.47) A d J , v . . * i f m ) 2 V d f t v . . * * J m ) . ( U i v . . .( Afm)*

Consequently,

(9.48) cp((/ - n - l A ) j l , ...( ( I - n P A ) f , )

1 2 d f t 9 * . . v $2 ivdf, 9 * * * 9 1,) . ( AflI . * 9 A f m )

1 2 d j l v * - * 9 f m ) - i A d j I , * . . 9 $ m L


and hence

or equivalently

for g , , . . . , g, E &(I - n - 'A). Since all convex functions on Rm can be approximated uniformly on any compact set K c R"' by a polynomial that is convex on K, (9.50) holds for all cp convex on 88"'.

Let F be given by (9.45), and define cp: Rm+ W by

(9.51)

Note that cp is convex. Then

5 IlFll. Similarly -AF = A(-F) s, 11 -Fll = IlFll, and the existence of the desired u follows.

There exists a transition function ~ ( x , r) such that

(see Appendix 8). and hence

(9.54) L q ( x . E M d x ) = v(E x E ) = p(E), B E B(E).

9. STATIONARV MSTRII)UTIONS 251

Let Y(O), Y(1), Y(2), . . . be a Markov chain with transition function v and initial distribution p. By (9.54). { Y(k)} is stationary. Since (9.44) holds for all h E C(E) and g E a(/ - n - ’ A ) , i t follows that

g(y)q(x, dy) = ( I - n - ’ A ) - l g ( x ) p a s . s (9.55)

for all g E @(I - n ’- ‘ A ) . Therefore

(9.56) h - l

is a martingale with respect to {.%:}. Let V be a Poisson process with parameter n and define X = Y( V(. )). Then

(9.57)

is an (4Fr}-martingale for each g G 91(1 - n - ’ A ) (cf. (2.6)). We leave i t to the 0 reader (Problem 41) to show that X is stationary.

Proposition 9.2 and Theorem 9. I7 are special cases of more-general results. Let A c B(E) x B(E). If X is a solution of the martingale problem for A and v, is the distribution of X(t ) , then ( v , } satisfies

(9.58)

where v, f = f dv,. Of course (9.40) is a special case of (9.58) with v , = p for all I 2 0. We are interested in conditions under which, given y o , (9.58) determines v, for all t 2 0. The first result gives a generalization of Proposition 9.2 (c - a).

V , ,f + vo .f + V, g d-5, (.L 9) E A, s.‘

9.18 Proposition Suppose *(A - A ) is separating for each 2 > 0. If { v , } and { p , } satisfy (9.58). are weakly right continuous, and v o = p o , then v , = p, for all t 2 0.

Proof. By (9.58), for (5 g) E A,

(9.59) 1 p%, f dl = vo f + ). r e - ” l v s q ds dt

= v,-, f + R

= vo f +

[me-af dt v,g ds

m

e -“vSg ds.

252 CENERArORS AND W K O V PROCESSES

Consequently,

Since W ( l - A ) is separating, (9.60) implies that vo uniquely determines the measure e-"v, dt. Since this holds for each I > 0 and {v,} is weakly right continuous, the uniqueness of the Laplace transform implies vo determines v, , t 2 0. 0

We next consider the generalization of Theorem 9.1 7

9.19 Proposition Let E be locally compact and separable, and let A be a linear operator on C ( E ) satisfying the positive maximum principle such that 9 ( A ) is an algebra and dense in QE). Suppose the martingale problem for A is well-posed. If {v , } c B(E) and {p,} c 9(E) satisfy (9.58) and vo = po, then v, = p, for all t 2 0.

Proof. Since 9 ( A ) is dense in &?), weak continuity of {v , } and {p,} follows from (9.58). We reduce the proof to the case considered in Theorem 9.17. As in the proof of Theorem 9.17, without loss of generality we can assume that E is compact. Let Eo = E x { - 1, 1). Fix 1 > 0. For f , E 9 ( A ) and f 2 E

S({ - 1, I}), let f = f J 2 and define

By Theorem 10.3 of the next section, if the martingale problem for A is well-posed, then the martingale problem for B is well-posed. There the new component is a counting process, but essentially the same proof will give uniqueness here.

Define

(9.62)

Then p satisfies B'dp = 0 for all /E ka(E), and, since the linear extension of B satisfies the conditions of Theorem 9.17, p is a stationary distribution for B.

We claim there is only one stationary distribution for 8 . To see this, we observe that any solution of the DEo[O, 00) martingale problem for B is a strong Markov process (Theorem 4.2 and Corollary 4.3). Let { q l ) be the one- dimensional distributions for the solution of the DEo[O, 00) martingale problem for (B, vo x dl). Let (2, N) be any solution of the Dso[O, 00) martingale problem for E, and define to = inf { t > O:N(t ) = - I } and t = inf { t > to:

N(t) = I}. Then

(9.63) p{(z(o, N o ) ) E q = P{(z(th No)) e r, 11 + ~ C ~ ~ - , ( ~ ) X , , . ~ , I .

10. PERTURBATION RESULTS 253

Consequently

(9

= lim r - I

and uniqueness of the stationary distribution follows.

tion for B and uniqueness gives

rt,(r) ds, 1 - C O l

If ji is defined by (9.62) with { v , } replaced by { p , } , i t is a stationary distribu-

(9.65)

Since I > 0 is arbitrary and { v f } and (p , } are weakly continuous, it follows 0 that v, = p, for all t 2 0.

10. PERTURBATION RESULTS

Suppose that X I is a solution of the martingale problem for A , c B ( E , ) x B ( E , ) and that X, is a solution of the martingale problem for A , c E ( E , ) x B(E,). If X , and X z are independent, then ( X I , X,) is a solution of the martingale problem for A c B ( E l x E 2 ) x B ( E l x E,) given by

(10. I ) A = (fi f z r 91 f2 +/I gz): (/I $91) E A I 9 ( f z - ~ 2 ) E A z ) .

If uniqueness holds for A , and A, . and if ( I , 0) E A, , i = I , 2, then we also have uniqueness for A.

10.1 Theorem Let ( E l , rl), ( E 2 , r 2 ) be complete, separable metric spaces. For i = 1, 2, let A, c HE,) x B(E,), ( I , 0) E A , , and suppose that uniqueness holds for the martingale problem for A, . Then uniqueness holds for the martingale problem for A given by (10.1). In particular, i f X = ( X , , X , ) is a solution of the martingale problem for A and X , ( O ) and X 2 ( 0 ) are independent, then X, and X 2 are independent.

Proof. if and only if i t is a solution for

Note that X = (XI, X,) is a solution of the martingale problem for A

(10.2) A = { C C J , + IIfiII + lMJ2 + I f z l l + 1).

g1(/2 + 1IfzII + 1 ) + (/I + I I f ~ I I + 1)gZ): CS,, ai) E Ai, i = 1. I}*

so we may as well assumel;. 2 1 for all (L, g,) E A, , i = I , 2.


For each w E *E,) for which a solution Y of the martingale problem for ( A i , w ) exists, define q,(w, r, r) = P { Y(r) E r), r E g(E, ) . By uniqueness, qr is well-defined.

By Lemma 3.2 and by Problem 23 of Chapter 2, Y is a solution of the martingale problem for A, if and only if for every bounded, discrete { *fl}-stopping time r,

for all (5 g) E A, .

to {g,} defined on (a, S, P). For I-2 E @ E , ) with P{X,(O) E r,} > 0, define Let X = (XI, X,) be a solution of the martingale problem for A with respect

( I 0.4)

Then XI on (a,#, Q) is a solution of the martingale problem for A , , and hence (10.5) E C X ~ , ( X ~ ( ~ ) ) X ~ ~ ( X ~ ( O ) ) I = qi(vrl , t , r#’{Xz(O) E rz) where wr2(rl) = P {X,(O) E I-‘] I X J O ) E r,}.

For ( A , gi) E A,, i = 1,2, define

(1 0.6)

( 10.8)

For (f, g) E A l and any discrete {Y,}-stopping time r, (10.7) implies

( 10.9)

10. KUTlJRBATION RESULTS 255

Consequently X I on (a, 9, (2) is a solution of the martingale problem for A , , and uniqueness implies that &{X,(r) E r,} = gl(v, c, r,) where v(r) = EEXdX I (W.f2( x 2(O))IIEC-f2( x 2(0))1.

Note that v does not depend on T ~ , so

Next, defining

(10.11)

(10.10) implies O{X2(t) E r,} = g2(v2, t , r2), where

Consequently,

Since by uniqueness the distribution of X ,(t) is determined by the distribution of Xl(0), v 2 is uniquely determined by the distribution of (X,(O), X2(0)). Conse- quently, the right side of (10.13) is uniquely determined by the distribution of

0 (X,(O), X2(0)). The theorem now follows by Theorem 4.2.

Let I E E(E) be nonnegative and let p(x, r) be a transition function on E x O(E) . Define E on B(E) by

Let A c B(E) x B(E) be linear and dissipative. If for some R. > 0, B(E) is the bp-closure of g(R. - A), then B(E) is the bp-closure of - ( A + B)) where A + E = ( ( J g + E j ) : ( A 9) E A } . Consequently, Theorem 4.1 and Corollary 4.4 give uniqueness results for A + E. Also see Problem 3.

We now want to give existence and uniqueness results without assuming the range condition.

10.2 Propodtion Let (E, r) be complete and separable, let A c B(E) x B(Ex and let B be given as in (10.14). Suppose that for every u E 9 ( E ) there exists a solution of the DJO, 00) martingale problem for (A, v). Then for every u E B(E) there exists a solution of the DEIO, 00) martingale problem for ( A + B, u).

Proof. By the construction in (2.4) we may assume 1 is constant. Let Q = flFm1 (DEIO, 00) x [O, a))). Let (Xk, A& denote the coordinate

random variables. Define Yk = dX,, Al: 15 k) and Yk = o(Xi , A,: 12 k). By an argument similar to the proof of Lemma 5.15, there is a probability distribution on Q such that for each k, xk is a solution of the martingale problem for A, Ak is independent of a ( X , , ..., XI, A l , ..., A k - , ) and exponentially distributed with parameter 1, and for A , E Yk and A2 E

Def ine r , -O , r ,=~~ , , A , , a n d N ( t ) ~ k f o r r , 5 r < t , + , . N o t e t h a t N i s a Poisson process with parameter 1. Define

(10.16) X(1) a Xk+,(t - T k ) , Tk 5 f < r k + l 9

and 9, = Sf VS;. We claim that X is a solution of the martingale problem for A + B with respect to IS,}. First note that for (f, g) E A,

(10.17)

(I V Q) A Q + 1

f(xk + I((t T k ) A 7k+ I - rk ) ) + - d x k + - T k ) ) ds

is an {P,}-martingale. This follows from the fact that

10. PERTURDATION RESULTS 257

is an {F,J-martingale, as are

(10.20)

and

k = I

Adding(10.20) and (10.21) to(10.19) gives

( 10.22)

which is therefore an {.F,}-martingale.

S ( X ( 0 ) -f(X(O)) - ~ ( . X ( S ) I + mw)) ds,

10.3 Theorem Let A t C(E) x W E ) , suppose 8 ( A ) is separating, and let B be given by (10.14). Suppose the DEIO, m) martingale problem for A is well- posed, let P , E P(D,[O, 00)) denote the distribution of the solution for ( A , ax), and suppose x-+ P, is a Bore1 measurable function from E into .P(DEIO. m)) (cf. Theorem 4.6). Then the DE rz+[O, 00) martingale problem for C c B(E x H,) x E(E x Z+), defined by

/h, gh + I . ( . ) ( / ( y ) h ( . + 1) s -J(.)M.))P(., d y )

is well-posed.

10.4 Remark Note that if (X, N ) is a solution of the martingale problem for C. then X is a solution of the martingale problem for A + B. The componcnt

0 N simply counts the “new”Jumps.

Proof. If the martingale problem for A is well-posed, then by Theorem 10.1 the martingale problem for A, = {(jh, gh) : (J g) E A. h E B ( Z + ) } is well-posed (for the second component, N ( t ) E N(0)). ForfE B(E x Z+)define

(1 0.24)

Then C = A, + B,, and the existence of solutions follows by Proposition 10.2.

S(K k ) = 4 x 1 ( / ( ~ t k + 1) - f ( x 9 ~ ) ) P ( x , d ~ ) . s For f E B(E) define

(10.25) W / ( X ) = ~p~cf(x(~))ll

258 GENERATORS AND MARKOV ?ROCESSES

and note that {T(t)) is the semigroup corresponding to the solutions of the martingale problem for A. Let (Y , N ) be a solution of the DErl,[O, a) martingale problem for C.

Note that

(10.26)

is a nonnegative mean one martingale, and let Q e 9(Da[0, GO)) be determined by (10.27) Q { X ( r , ) E r, , ..., X(t , ) E rm}

for 0 2 r , < t , c < t,, rl , . . ., rm E a(E). (Here X is the coordinate process.) Since (Y, N) is a solution of the martingale problem for C, it follows by Lemma 3.4 that

is an {f~r*N'}-martingale for (f, g ) E A. Since (10.26) and (10.28) are martingales,

1 0

for t , < t , < - * < t,+ , , (J; g) E A, and hk E B(E), and it follows that Q is a solution of the martingale problem for A. In particular,

More generally, for r 2 s.

To complete the proof we need the following two lemmas.

10. PERTURMTION RESULTS 259

- A(y(u)) d' X(N(s t t n ) = N ( s ) ) d N ( s )

10.5 Lemma For f E B(E) and r 2 0,

-

(10.34)

0

260 CENEILATORS AND MARKOV PROCESSES

10.6 lemma For h E E(E) and f z 0,

Proof. For (S, 8) E A,

- f( Y(d)X(N(r) =&I) d Y(sh dy)) ds

is a right continuous martingale. Consequently, if r, = inf {t: N(r) = k},

(10.39) ECf(Y(~&))x(,.l,1 = E [~;:;4Y,s)) JJmO(s), dY) ds].

Summing over k gives (10.37) with h =J For general h, the result follows from 0 the fact that 9 ( A ) is separating.

Proof of Theorem 10.3 continued. From (10.30) and Lemmas 10.5 and 10.6,

( 10.40) ECf( Y(N1 - El: W f ( Y(0))l

11. PROllLEMS 261

that is.

(10.4 I ) a/( YW)I = E V - ( W ( Y(0))J

+ [ E [ B T ( r - s ) f ( Y ( s ) ) ] ds

for every/€ B(E). Iterating this identity gives

(10.42) EM w i = m w Y(O))I

+ [E[T(s)f?TO - s)/( Y(O))] ds

+ Jrl J)pr(s - u)Br( t - .qj(r(tl))l dll ds,

and we see that by repeated iteration the left side is uniquely determined in terms of Y(0) . {T(r)}, and B. Consequently, uniqueness for the martingale

0 problem follows by Theorem 4.2.

11. PROBLEMS

1. (a) Show that to verify (1.1) it is enough to show that

( 1 1 . 1 ) f { x ( u ) E rix(t,,),X(tn-l),..., x(t,)} = P { X ( U ) E rix(t)) for every finite collection 0 5 r 1 c t 2 c . . * < r, = t < u.

(b) Show that the process constructed in the proof of Theorem 1.1 is Markov.

2. Let X be a progressive Markov process corresponding to a measurable contraction semigroup { T(t)} on B(E) with full generator A'. Let A l , A * , . . . be independent random variables with P{A, > I ) = e -', t 2 0, and let V be an independent Poisson process with parameter 1. Show that X ( n - ' &"?\' Ak) is a Markov process whose full generator is A, = A(! - n-'&-', the Yosida approximation of A'.

3. Suppose { T(t)) is a semigroup on B ( E ) given by a transition function and has full generator A'. Let

( I 1.2) W ( X ) = 4 x ) (S(Y) - - f ( X ) ) P k dv) I where 1 E B(E) is nonnegative and p(x, r) is a transition function. Show that A'+ B is the full generator of a semigroup on B(E) given by a transition function.

262 GENERATORS AND WKOV PROCESSES

4. Show that X defined by (2.3) has the same finite-dimensional distributions as X defined by (2.6).

5. Dropping the assumption that A is bounded in (2.3), show that X ( t ) is defined for all t 2 0 with probability I if and only if P { c ; f i o l/A(Y(k)) = 00) = 1. In particular, show that

6. Show that X given by (2.3) is strong Markov. 7. Let X be a Markov process corresponding to a semigroup { T(t)} on RE).

Let V be an independent Poisson process with parameter 1. Show that X( V(nt)/n) is a Markov process. What is its generator?

8. Let E = (0, 1, 2, ...}. Let qij 2 0. i #/, and let I,*, qu = -qrr < 00. Suppose for each i there exists a Markov process X' with sample paths in D,[O, 00) such that X'(0) = i and

(1 1.3)

P({X?=O A k / 4 W ) ) = 4 A{X?=o 1 / 4 Y ( k ) ) =4) = 0.

Iim &-l (P{X' ( t + E ) = j l X'(r)} - x,,,(x'(t))) r - O +

= qxccr)lr j E E, t L 0.

(a) Show that X' is the unique such process. (b) For i E E and n = I , 2, . . . , let Xi be a Markov process with sample

paths in D,[O, 00) satisfying Xf(0) = i and

( I I .4) Iim E - '(P{ xi(t + E ) =I j I x:(t)) - x,,,(xi(t))) r - 0 +

= 4 X , W , (n) j E E, t 2 0.

Show that Xi

( I 1.5) lim 4:;) = qu, i, j E E

(cf. Problem 31).

X' for all i E E if and only if

n- m

9. Prove Theorem 2.6.

10. Let e l , t t , , . . be independent, identically distributed random variables with mean zero and variance one. Let

(11.6)

Show that X n * X where X is standard Brownian motion. (Apply Theorem 6.5 of Chapter 1 and Theorem 2.6 of this chapter, using the fact that CF(R) is a core for the generator for X.)

11.

12.

13.

14.

15.

Let Y be a Poisson process with parameter I and define

(1 1.7)

Use Theorem 6.1 of Chapter I and Theorem 2.5 of this chapter to show that {X,,} converges in distribution and identify the limit.

Let E = R and A / ( x ) = a(x)f"(x) + b(x ) f ' (x ) for/€ CF(R), where a and 6 are locally bounded Bore1 functions satisfying 0 s a ( x ) 5 K(l + I x 12) and xb(x) s K( l + 1x1') for some K > 0. Show that A is conservative. Extend this result to higher dimensions.

Let E = W and Af (x ) = x2(sin2 x ) f ' ( x ) and Sf=/' for/€ C:(R).

(a) Show that A, B, and A + B satisfy the conditions of Theorem 2.2, (b) Show that A and B are Conservative but A + B is not.

X,,(r) = n- I ( Y(n2t) - An't).

Complete the proof of Lemma 3.2.

Let E be locally compact and separable with one-point compactification EA. Suppose that 1 E M(E) is nonnegative and bounded on compact sets, and that p(x, r) is a transition function on E x a(€). Define X as in (2.3). setting X(f) = A for t 2 ~~~o AJ1( Y(k)), and let

/A(x) I ( f ( y ) -/(x))p(x, dy) x E E

for eachf E B(EA) such that

(1 1.9)

(a) Show that X is a solution of the martingale problem for A. (b) Suppose B(EA) is the bp-closure of the collection offsatisfying (1 1.9).

Show that X is the unique solution of the martingale problem for ( A , v ) , where v is the initial distribution for Y, if and only if

(c) Let E = Rd. Suppose sup, A(x)p(x, r) < og for every compact

SUP 4 x ) l f ( Y ) --/(x)Ip(x9 dY) < s x e E

f{Z& l/A(Y(k)) =a} = 1.

r c Rd, and

(11.10) I ( x ) I y - x ( p ( x , d y ) s K ( l + I x l ) , XERd, I for some constant K . Use Theorem 3.8 to show that X has sample paths in D,[O, a), and show that X is the unique solution of the martingale problem for (A, v).


16. (Discrete-time martingale probIem) (a) Let ~ ( x , r) be a transition function on E x I ( E ) and let X(n), n = 0,

1, 2, ... be a sequence of E-valued random variables. Define A : E(E) - r B(E) by

(11.11) A f M = Jl(Y)P(X, dY) -Ax),

and suppose

(1 1.12)

is an {Sf)-martingale for each/€ E(E), Show that X is a Markov chain with transition function p(x, r).

(b) Let X(n), n = 0, 1,2, . . . , be a sequence of 2-valued random variables such that for each n 2 0, 1 X(n + 1) - X(n) 1 = 1. Let g : Z -+ [ - 1, 13 and suppose that

n-1

k 5 0 / ( X ( n ) ) - c Af(X(k))

X(n) - "i'eCXCk)) k - 0

is an (Pf)-martingale. Show that X is a Markov chain and calculate its transition probabilities in terms of g.

17. Suppose that (E, r ) is complete and separable, and that P(r, x, r) is a transition function satisfying

(11.13)

for each E > 0. (a) Show that each Markov process X corresponding to f i r , x, T) has a

version with sample paths in DBIO. 00). (Apply Theorem 8.6 of Chapter 3.)

lim sup P ( ! , x, ~ ( x , ey) = o #+a r

(b) Suppose

( I 1.14)

for each E > 0. Show that the version obtained in (a) has sample paths as. in C,[O, a) (cf. Proposition 2.9.).

tim sup n ~ ( t , x, ~ ( x , ey) = o n-to x

18. Let E be compact, and let A be a linear operator on C(E) with 9 ( A ) dense in C(E). Suppose there exist transition functions p,,(x, r) such that for each/€ 9 ( A )

( I 1.15) n-m

uniformly in x and that

(1 1.16)

for each E > 0. Show that for every v E 9 ( E ) there exists a solution of the martingale problem for (A, v ) with continuous sample paths.

Let (E , r ) be separable, A c B(E) x B(E), J g E M ( E x E), and suppose that for each y E E, (f(., y), g ( * , y)) 6 A. If for each E > 0 and compact K c E, inf ( J ( x , y) - f ( y . y ) : x , y E K, r(x, y ) 2 E } > 0 and if for each x E E, g(x, y ) = g(x, x) = 0, then every solution of the martingale problem for A with sample paths in Dg[O, 03) has almost all sample paths in C,[O, 03) (cf. Proposition 2.9 and Remark 2.10).

20. For i = I , 2, . . . , let E, be locally compact and separable, and let A, be the generator of a Feller semigroup on C(E,) . Let E = n;.=, E , . For each i , let p, E B(E) be nonnegative. For g(x) = n7ml J(x,) , n 2 I , 1;. E 9 ( A i ) , define

lim sup n p#(x, B(x, E Y ) = 0 n+oD x

19.

(11.17)

Show that every solution of the martingale problem for A has a modification with sample paths in DJO, a).

21. Let E be the set of finite nonnegative integer-valued measures on (0, I , 2, ...} with the weak topology (which in this case is equivalent to the discrete topology). For/’€ E(E) with compact support, define

k 2 ( f ( a + 6, - , - 6,) - f (a) )a(dk) . ( I 1.18)

(a) Interpret a solution of the martingale problem for A in terms of particles moving in (0, I , 2, .. .).

(b) Show that for each v E B(E), the DEIO, 03) martingale problem for (A, v ) has a unique solution, but that uniqueness is lost if the requirement on the sample paths is dropped.

A/(a) = I

22. Let E = LO, 13 and A = { ( J - j ’ ) : f ~ C’(E),f(O) = j ( l ) ] .

(a) Show that A satisfies the conditions of Corollary 4.4. (b) Show that the martingale problem for ( A , his more than one

solution if the requirement that the sample paths be in DE[O, m) is dropped.

23. Use (4.44) to compute the moments for the limit distribution for X. Hint: constant.

Write the integral as a sum over the intervals on which Y is

266 GENERATORS AND MARKOV PROcfSsEJ

24. Let E l = [0, 1) and A , = {(f, x(l - x ) f " +(a - bx)f'): f E C2(E,)), where 0 < a < b. Use duality to show uniqueness for the martingale problem and to show that, if X is a solution, then X(r) converges in distribution as t -+ 00 with a limiting distribution that does not depend on X(0) . Hint: Let E, = (0, 1,2, ...} andf(x, y) = xy.

25. Let E l = El = [O, a), A , = { (A 9"): f E c2(El), f"(0) = 0}, and A, = {(J tf"): f e C2(E2), f'(0) = 0}, that is, let A, correspond to absorbing Brownian motion and A, to reflecting Brownian motion. (a) Let g E cz(-03, 03) satisfy g(z) = -g(-z). Show that the martin-

gale problems for A , and A2 are dual with respect to (S, 0, 0) where f(x, Y) = e(x + Y) + Q(X - Y).

(b) Use the result in part (a) to show that P ( X ( t ) > y I X(0) -- x} = P{ Y(t) < x I Y(0) = y } , where X is absorbing Brownian motion and Y is reflecting Brownian motion.

26.

27.

28.

Let E = [0, 13, A = {(S, if"): f E C'(E),j'(O) = f ' ( l ) = f"(0) =f"(I) = 0}, and let r c B(DJ0, 00)) be &,,, the collection of solutions of the martingale problem for A.

(a) Show that r satisfies the conditions of Theorem 5.19. (b) Find a sequence { f k } , as in the proof of Theorem 5.19, for which

p m ) = A,,,, where

A , = { (A f f"):fE C2(E),/"(O) =f"(l) = 0).

(c) Find a sequence (X} for which P" = AA2 , where

A, = {(S, f f " ) : f ~ C2(E),f'(0) =r(l) = 0).

Let U k , k J 1 ,2 , . .., be open with E = uz, vk. Given x c DEIO, a), let S(, = inf { u 2 1 : x ( u ) $ U, or x ( u - ) f? uk}. Show that there exists a sequence of positive integers k , , k,, . . . and a sequence 0 = t 1 < r , < - such that t i+ , = S:, for each i 2 1 and limi-m t, = 00. In particular every bounded interval [0, T] is contained in a finite union UfI , [ t i , St,). Hint : Select k - ( t ) so that ~ ( t - ) E uk-(,) and k + ( t ) so that x(r) E Oh'((), and note that there is an open interval I, with t E I , such that { x ( s - ) ,

Let ( E L , rk), k = 0, I , 2, ..., be complete, separable metric spaces. Let E = u k EI, (think of the Ek as distinct even if they are all the same), and define r(x. y ) = rk(x, y ) if x, y E Ek for some k , and r(x, y ) = 1 otherwise. (Then ( E , r ) is complete and separable.) For k = 0, I, 2, . . . , suppose that A, t f(f$) x t ( E k ) , that the closure ofgenerates a strongly continuous contraction semigroup on Lk 3 g(Ak), that Lk is separating, and that for each v E g(&) there exists a solution of the martingale problem

x(s) : s E f , ) c uk-(,) u uk+(,).

11. PROBLEMS 267

for ( A k , v) with sample paths in D,,[O, ao). Let A c c(E) x < f ( E ) be given by

(a) Show that the closure o-enerates a strongly continuous contraction semigroup on L E 9 ( A ) .

(b) Show that the martingale problem for A is well-posed. (c) Let A E C(E), A 2 0, and supx. €, A(x) < 00 for each k. Let p(x, r) be

a transition function on E x B(E) and define

( 1 1.20) W ( x ) = 4 x ) (J(Y) - J ( x ) ) p ( x . 44 s for/€ c(E). Suppose B c C(E) x C(E).

Let

(11.21) Ak, = sup j(x)p(x, El) x e €1

and suppose for some a, b 2 0,

( 1 1.22) 1 5 a -t bk, k 2 0.

Show that for each v E B(E) there exists a unique solution of the local-martingale problem for ( A + B, v ) with sample paths in

Remark. The primary examples of the type described above are population models. Let S be the space in which the particles live. Then E, = SL corresponds to a population of k particles, A,, describes the motion of the k particles in S, and I and p describe the reproduction.

29. Let A be given by (7.15). let A satisfy the conditions of Lemma 7.2, and define

I > k

D € P , a).

( I I .23)

where P is the solution of (7.6).

(a) Let X be a solution of the martingale problem for A. Show that

( 1 I .24) Us, r ) . f ( W s ) ) , 0 5 s 5 r

is a martingale. (b) Show that the martingale problem for A is well-posed.

2611 GENERATORS AND maov PROCESSES

30. Let C1, C, , . . . be a stationary process with EL&+ I I tl , t3, . . . , t k ] = 0. Suppose €[ti] = oz and = 0’ 8.5. Apply Theorem 8.2 to show that { X,} given by

n - ‘ zml

(1 1.25)

converges in distribution. Hint: Verify relative compactness by estimating E[(X,(t + u) - X,(t))’ IS3 and applying Theorem 8.6 of Chapter 3.

31. (a) Let (E, r) be complete and separable. Let IT&)}, n 5: 1, 2, .. ., and { T(t)} be semigroups corresponding to Markov processes with state space E. Suppose that T(t): L c 415)- L, where L is convergence determining, and that for each/€ L and compact K c E,

(1 1.26) lim sup I q(r)(t)/(x) - T(t)f(x) I,

Suppose that for each n, X, is a Markov process corresponding to { T,( t )} , X is a Markov process corresponding to (T(r)}. and X,(O) * X(0). Show that the finite-dimensional distributions of X, converge weakly to those of X.

(b) Let E = (0, 1,2, ...}. For/€ B(E),define

I 2 0. n-m x e K

0 k - 0

k # O , n n(/(O) -S(n)) k = n,

Show that TAr) i efAa and T(t) I kA satisfy (1 1.26). Fix k > 0. For each n 2 1, let X , be a Markov process corresponding to A, defined in (b) with X,(O) = &. Show that the finite- dimensional distributions of X, converge weakly but that X , does not converge in distribution in D,[O, 00).

32. Let E be locally compact and separable, and let {T(t)j and {S(r)] be Feller semigroups on t ( E ) with generators A and E, respectively. Let { x ( k ) , k = 0, 1, .. .} be the Markov chain satisfying

(c)

ECf(Yn(2k + 1)) I Y J ~ ~ ) J = T ( ’) n j(Yn(2k))

and

ECf(Yn(2k)) I ~ , ( 2 k - 111 = s (i) J ( Y , ( z ~ - 1)).

11. PROBLEMS 269

and set X , ( f ) = V,([ntJ). Suppose that 9 ( A ) n S(B) is dense in t ( ( E ) .

Show that {X,} is relatively cornpact in DEb[0, OD) (E" is the one-point compactification of E), and that any limit point of {X,] is a solution of the martingale problem for A" + BA ( A A and BA as in Theorem 5.4).

33. Consider a sequence of single server queueing processes. For the nth process, the customer arrivals form a Poisson process with intensity A,,, and the service time is exponentially distributed with parameter p, (only one customer is served at a time). (a) Let V, be the number of customers waiting to be served. What is the

generator corresponding to V,? (b) Let X,(t) = n-"2Yn(nc). What is the generator for X,?

(c) Show that if {X , (O) } converges in distribution, limn-,m 1, = 1, and n(A , - 1,) = a, then {X,) converges in distribution. What is

the limit? (d) What is the result analogous to (c) for two servers serving a single

queue? (e) What if there are two servers serving separate queues and new

arrivals join the shortest queue? Hint: Make a change of variable. Consider the sum of the two queue lengths and the difference.

34. (a) ' Let €, , t2, . . . be independent, identically distributed real random variables. For x,, , a E R, let Y,(O) = xo and

( I 1.27)

ForfE Cf(W), calculate

(I 1.28) lim n E [ f ( Y , ( k + I ) ) -f(Y,(k))l Y,(k) = XI,

and use this calculation to show that X,, given by X , ( t ) = Y,([nt]) , converges in distribution.

(b) Generalize (a) to d dimensions.

Let E be locally compact and separable, let 1: &--. [0, 00) be measurable and bounded on compact subsets of E , and let p(x, r) be a transition function on E x @ E ) . Forftz C,(E), define

Y,(k + I ) = ( I + n - 'a)Y,(k) + n - "'t., + , , k = 0, I , . . . .

,-.a0

35.

A m = 4 x ) ( f ( Y ) - . f (X))P(.X* dY). 5 ( I 1.29)

Let v E B(E) , and suppose that the local-martingale problem for ' ( A , v ) has a solution X with sample paths in D,[O, 00) (i.e., does not reach infinity in finite time). Show that the solution is unique.

(a)

270 CENfUTORS AND MAIKOV MOCESSES

(b) Suppose that rp and I (p(y)lp(., dy) are bounded on compact sets. Suppose that X is a solution of the local-martingale problem for (A, w ) with sample paths in Ds[O, 00). Show that

(1 1.30) rp(X(t)) - &I(XO) I(rp(u) - cp(X(s)Mo(s), dv) ds

is a local martingale. In addition to the assumptions in (b), suppose that (p L 0 and that there exists a constant K such that

(1 1.31)

for all x. Show that (1 1.30) is a supermartingale and that

(c)

4 x ) (dY) - Cp(X))P(X, dY) s K I (1 1.32) cp(X(t)) - K t

is a supermartingale.

36. Let A c e ( E ) x c((E). Suppose that the martingale problem for A is well- posed, that every solution has a modification with sample paths in DEIO, GO), and that there exists xo E E such that every solution (with sample paths in DEIO, m)) satisfies T = inf ( L : X(t ) = xo) < Q, as. Show that there is at most one stationary distribution for A.

37. Let E = R, a, 6 E C2(E), a > 0, and A = {(J af” + bf’): /E C:(E)}. Suppose there exists g E Cz(E), g 2 0, satisfying

(1 1.33)

and f?- g d x = 1. Show that if the martingale problem for A is well- posed, then g is the density for the unique stationary distribution for A.

38. Let E = 88 and A = { ( f i f ” + x ” f ’ ) : I ~ C,“(E)}. Show that there exists a stationary solution of the martingale problem for A and describe the behavior of this process.

39. Let E = [O, I], a, b E C(E), a > 0, and A = {(A uf” + bf’): f~ Cm(E), f’(0) =I1(]) = 0). Find the stationary distribution for A.

40. Let E = [O, 11, and A = { (A if”): f~ C2(E), f’(0) =f’(l) = 0, and f‘(# =f’(i))}. Show the following: (a) = c(E).

(b) @(A - A) # C(E) for some (hence all) R > 0.

(c) The martingale problem for A is well-posed.

11. ~ O @ I L E M S 271

(d) p ( r ) = 3m(r n [f., 33) (where m is Lebesgue measure) satisfies

(1 1.34) iJf’ dp = 0, S E OW,

but p is not a stationary distribution for A.

41. Show that X defined in the proof of Theorem 9.1 7 is stationary. 42. Let (E, r ) be complete and separable. If X is a stationary E-valued

process, then the ergodic theorem (see, e.g., Lamperti (1977). page 102) ensures that for h E E(E),

(1 1.35)

exists a.s. (a) Let Y E e€). Show that if ( I 1.35) equals I h dv for all h E C(E), then

this equality holds for all h E W E ) . (b) Let P(r, x, r) be a transition function such that for some v E P(E)

( 1 1.36) lim r - ’ 1 h(y)P(s, x, dy) ds = h dv, x E E, h E C(E).

Show that there is at most one such v. Let Y and f i t , x, r) be as in (b). Let X be a measurable Markov process corresponding to or, x, r) with initial distribution Y (hence X is stationary). Suppose that ( 1 1.35) equals It dv for all h E E(E). Show that X is ergodic. (See Lamperti (1977). page 95.)

Let (E, r ) be complete and separable. Suppose P(r, x, r) is a transition function with a unique stationary distribution Y E a€). Show that if X i s a measurable Markov process corresponding to P ( r , x, r) with initial distribution v, then X is ergodic.

14. For n = I , 2, . .., let X, be a solution of the martingale problem for A, = { ( J J ” + nb(n.)f’):fE C,‘(lra)}, where h is continuous and in L‘, and let a = JTrn b(x) dx. Let X be a solution of the martingale problem for A with

9 ( A ) = {/E C,(R):f’ and f” exist and are continuous except

lim t - ’ fi(X(s)) ds r -m

1-m s (c)

43.

at zero, J’(0 + ) = e -”/‘(O - ), and j ” ( 0 + ) = /” (0 - )},

and AJis the continuous extension off ”. (a) Show that uniqueness holds for the martingale problem for A. (b) Show that if XJO) =- X(O), then X, r+ X.

Hint: apply the results of Section 8.

For /E 8 ( A ) , let S. satisfy f . ” ( x ) -k nb(nx)J;(x) = Af(x), and

(c) What happens if i2'- b'(x) dx = 00 and J?, 67x) dx 2 00? 45. Let (E, r) be complete and separable and let A c C(E) x B(E), Suppose

9 ( A ) is separatind. Let X be a solution of the Ds[O, ao) martingale problem for A, let r E I@), and suppose B(x) = 0 for every x E r and (f, 8) E A. For t 2 0, define y, = in€ {u > t : fi xP(X(s)) ds > 0). Show that X(u hy,) = X ( t ) as. for all u > t , and that with probability one, X is constant on any interval [t, u] for which fi x,(X(s)) ds = 0.

16. Let E be separable. For n = 1, 2, ..., let {I#), k = 0, 1, 2, ...} be an &-valued discrete-time stationary process, let em > 0, and assume en+ 0. Define Xn(t) = Y.([t/q,]), and suppose X , 3 X. Show that X is stationary.

47. Let E be locally compact and separable but not compact, and let Ed = E u {A} be the one-point compactification. kt v(x, r) be a transition function on E x a@), and let rp, J, E M(E). Suppose that cp 2 0, that $ s C for some constant C and limx-,A $(x) = -00, and that for every Markov chain { Y(k), k = 0, 1,2, . . .) with transition function v(x, r) satisfying E[q( Y(O))] < 00,

(11.37) P(Y(4) - kil $(Y(O) 1-0

is a supermartingale. Suppose Y is stationary. Show that

m P { Y(0) E K,} 2 - C + m '

where K, = {x: $(x) 2 -m}.

48. For i = I , 2, let E, be a locally compact (but not compact) separable metric space and let E f = E, u (A,} be its one-point compactification. Let X be a measurable E,-valued process, let Y be a measurable &,-valued process, and let cp E M ( E , ) and # E M(E, x E2). For t > 0, define

and

Suppose that PI

12. NOTES 273

is a supermartingale, cp 2 0, # 5 C for some constant C, and that for each compact K2 c E,, lim,,,, sup,,. K 1 $(x, y) = - 00. Show that if ( v , : r 2 1) is relatively compact in 9(E2), then {p , : r 2 I } is relatively compact in 9(€,). (See Chapter 12.)

49. (a) Let E be compact and A c C(E) x C(E) with 9 ( A ) dense in C(E). Suppose the martingale problem for (A, 6,) is well-posed for each x E E. Show that the martingale problem for A is well-posed (i.e., the martingale problem for (A, p ) is well-posed for each p E 9 ( E ) ) .

(b) Extend the result in (a) to E and A satisfying the conditions of Theorem 5.1 I(b).

50. Let E l = E , = [O , 1 J, and set E = El x E,. Let

A , = { (A sz*J; J2):.f2 E C~E,),Sl E CZ(E1)r/;(0) =

A, = {(I1 XI01 9 J’; X,& 0 E E , 1 h E C2(E, 1, J-;m =

(1 1.41) f’;(O) - f ; ( l , =J; (1 ) = 0) .

f ; ( l ) = 01.

and A = A l u A , . Show that the martingale problem for ( A , #,,,,,) is well-posed for each (x, y) E E but that the martingale problem for (A, p ) has more than one solution if fi is absolutely continuous (cf. Problem 26).

12. NOTES

The basic reference for the material in Sections I and 2 is Dynkin (1965). Theorem 2.5 originally appeared in Mackevicius (1974) and Kurtz ( 1 975).

Levy (1948) (see Doob (1953)) characterized standard Brownian motion as the unique continuous process W such that W(r) and W(t) , - 1 are martingales. Watanabe (1964) characterized the unit Poisson process as the unique counting process N such that N ( t ) - r is a martingale. The systematic development of the martingale problem began with Stroock and Varadhan (1969) (see Stroock and Varadhan (1979)) for diffusion processes and was extended to other classes of processes in Stroock and Varadhan (1971). Stroock (1975), Anderson (1976). Holley and Stroock (1976, 1978).

The primary significance of Corollary 4.4 is its applicability to Ray processes. See Williams (1979) for a discussion of this class of processes. Theorem 4.6 is essentially Exercise 6.7.4 of Stroock and Varadhan (1979).

The notion of duality given by (4.36) was developed first in the context of infinite particle systems by Vasershtein ( 1 969). Vasershtein and Leontovitch (1970), Spitzer (1970), Holley and Liggett (1975). Harris (1976), Liggett (1977), Holley and Stroock (1979). It has also found application to birth and death processes (Siegrnund (1976)). to diffusion processes, particularly those arising

274 GENERATORS AND MMKOV ruocEssps

in genetics (Holley, Stroock, and Williams (1977), Shiga (1980, 1981), Cox and Rosler (1982)) (see Problem 29, and to measure-valued processes (Dawson and Kurtz (1982), Ethier and Kurtz (1986)).

Lemma 5.3 is due to Roth (1976). Theorem 5.19 is a refinement of a result of Krylov (1973). The presentation here is in part motivated by an unpublished approach of Gray and Griffeath (1977b). Set also the presentation in Stroock and Varadhan (1979).

The use of semigroup approximation theorems to prove convergence to Markov processes began with Trotter (1958) and Skorohod (1958). although work on diffusion approximations by Khintchine (1933) is very much in this spirit. These techniques were refined in Kurtz (1969, 1975). Use of the martingale problem to prove limit theorems began with the work of Stroock and Varadhan (1969) and was developed further in Morkvenas (l974), Papanicol- aou, Stroock, and Varadhan (1977), Kushner (1980), and Rebolledo (1979) (cf. Theorem 4.1 of Chapter 7). Proposition 8.18 abstracts an approach of Helland (1981). The recent book of Kushner (1984) gives another development of the convergence theory with many applications.

The results on existence of stationary distributions are due to Khasminskii (1960, 1980), Wonham (1966), Benes (1968), and Zakai (1969). Similar convergence results can be found in Blankenship and Papanicolaou (1978). Cos- tantini, Gerardi, and Nappo (1982), and Kushner (1982). Theorem 9.14 is due to Norman (1977). Theorem 9.17 is due to Echeverria (1982) and has been extended by Weiss (1981).

Problem 25 is from Cox and Rosler (1982). Problem 40 gives an example of a well-posed martingale problem with a compact state space for which the closure of A is not a generator. The first such example was given by Gray and Grifleath (1977a). Problem 44 is due to Rosenkrantz (1975).

5

The emphasis in this chapter is on existence and uniqueness of solutions of stochastic integral equations, and the relationship to existence and uniqueness of solutions of the corresponding martingale problems. These results comprise Section 3. Section I introduces d-dimensional Brownian motion, while Section 2 defines stochastic integrals with respect to continuous, local martingales and includes 1"s formula.

STOCHASTIC INTEGRAL EQUATIONS

1. BROWNIAN MOTION

Let r , , C z , . . . be a sequence of independent, identically distributed, R'-valued random variables with mean vector 0 and covariance matrix I d , the d x d identity matrix. Think of the process

as specifying for fixed ti 2 1 the position at time t of a particle subjected to independent, identically distributed, random displacements of order I/& at times I/n, 2/n, 3/n, . . . . Now let n-, Q). In view of the (multivariate) central limit theorem, the existence of a limiting process (specified in terms of its finite-dimensional distributions) is clear. If such a process also has continuous sample paths, i t is called a d-dimensional Brownian motion.

275



276 STOCHASTIC INTEGRAL EQUATIONS

More precisely, a process W = (W(t), t 2 0) with values in R“ is said to be a (standard) d-dimensional {.!F,)-Brownian motion if:

(a) W(0) = 0 as. (b) W is adapted to the filtration {SJ, and 9, is independent of

(c) W(t) - W(s) is N ( 0 , ( t - s)lJ (ie., normal with mean vector 0 and

(d) W has sample paths in Cw[O, a).

a( W(u) - W(r): u 2 t ) for each t 2 0.

covariance matrix (t - s)l,) for every r > s 2 0.

When (9,) = {PY} in the above definition, W is said to be a (standard) d-dimensional Brownian motion.

Note that if W is a d-dimensional {9,}-Brownian motion defined on a probability space (Q, 9, f), then W is a d-dimensional (#,}-Brownian motion on (n, $, f), where #, and .# denote the P-completions of PI and 9, and P denotes its own extension to 9. (If 49 is a sub-a-algebra of 9, the P- complction of $ is defined to be the smallest o-algebra containing Y u ( A c Q: A c N for some N E f with P ( N ) = O}.)

The existence of a d-dimensional Brownian motion can be proved in a number of ways. The approach taken here, while perhaps not as eficient as others, provides an application of the results of Chapter 4, Section 2.

We begin by constructing the Feller semigroup { T(t)} on t(R’) corresponding to W. The interpretation above suggests that { T(r)) should satisfy

for all/€ e(W’), x E W’, and t 2 0. By the central limit theorem, (1.2) is equivalent to

(1.3) T(t)/(x).= + &)I, where 2 is N(0, I,). We take (1.3) as our definition of the semigroup (T( t ) } on e( 88,).

1.1 Proposition Equation (1.3) defines a Feller semigroup { T(t)} on e(Rd). Its generator A is an extension of

where A, E z- 8:. Moreover, CF(R‘) is a core for A.

1. BROWNIAN MOTION 277

Proof. For each f L 0, T(t): e(Rd)-+ e((Qgd) by the dominated convergence theorem, so T(t) is a positive linear contraction on c(R“). Let Z be an independent copy of Z. Then, by Fubini’s theorem,

(1.5) T(W(t)f(-x) = E [ T ( t ) f ( x + &)I = E [ f ( x + &Z + 4231

= T(s + t ) f ( x )

= E [ j - ( x + G Z ) ]

for all/€ c([wd), x E Rd, and s, I z 0. Since T(0) = I , this implies that { T(t ) } is a semigroup. Observe that each le c(Rd) is uniformly continuous (with respect to the Euclidean metric), and let w(J S) denote its modulus of continuity, defined for 6 2 0 by

(1.6)

Then llT(t)f-j’ll 5 E [ w ( J d Z ) ] for all t 2 0, so by the dominated convergence theorem, { T(t)} is strongly continuous.

To show that the generator A of (T( t ) ) extends (1.4). f ix f~ ez(08’). By Taylor’s theorem,

(1.7)

wcj, f i ) =sup { IS(Y) -mI: x, y E Rd, Iv - X I 5 6) .

T(0ftx) - f ( x ) = ECf(x + J;z) -/(x)I I d

1 = I

We conclude thatfE 9 ( A ) and AJ- +Ad& Observe next that (1.3) can be rewritten as

(1.9) T ( i ) / ( x ) = & I ( Y H ~ ~ ) - ” ~ ~ exp { - I y - x lz/2t} dv.

provided I > 0. I t follows easily that

278 STOCHASTIC INTEGUL EQUATIONS

(1.10) T(2): C(Wd)+ c-(wl,, f > 0.

where em(@) = ( Ik2 , &@). By Proposition 3.3 of Chapter I , @(Wd) is a core for A. Now choose h E CF(iRd) such that x ~ ~ : ~ ~ ~ ~ ,, s h 5 x , ~ : lx lr21 , and define (A,) c CT(Rd) by h,(x) = h(x/n). Given f IS &Wd), observe thatJh,+f and A( f h,) = (Af)h, +/Ah, + Vf - Vh,+ Af uniformly as n-, 00, implying that CF(Rd) is a core for A. Finally, since bp-lim,,, (b,, Ah,) = (I , 0), A is conservative. 0

The main result of this section proves the existence of a d-dimensional Brownian motion and describes several of its properties.

1.2 Theorem A d-dimensional Brownian motion exists. Let W be a d- dimensional (9,)-Brownian motion. Then the following hold:

(a1 W is a strong Markov process with respect to {S,} and corresponds to the semigroup { T(r)) of Proposition 1.1.

(b) Writing W = (W, , ..., Wd), each H( is a continuous, square- integrable, (9,)-martingale, and (%, W,), = 6,,t for i, j = 1, . . . , d and all t 2 0.

(c) With ( X , ) defined as in the first paragraph of this section, X, =+ W in &[O, a) as n-+ 00.

Proof. By Proposition 1.1 of this chapter and Theorem 2.7 of Chapter 4, there exists a probability space (a, P, P) on which is defined a process W = { W(t), I 2 0 ) with W(0) = 0 as. and sample paths in Dnr[O, 00) satisfying part (a) of the theorem with {P,} = {ST}. In fact, we may assume that W has sample paths in Cw[O, 00) by Proposition 2.9 of Chapter 4. For r > s 2 0 and

/ E @Fitd), the Markov property and (1.3) give

Consequently, since W(s) is 9;?rpmeasurabk,

It follows that 9: is independent of a(W(u) - W(s): u L s) and that W(t) - W(s) is N(0 , (r - s)ld) for all t 1 s L 0. In particular, W is a d- dimensional Brownian motion.

Let W be a d-dimensional {.F,}-Brownian motion where (9,) need not be (9,'"). To prove part (a), let 7 be an (9,}-stopping time concentrated on ( t , ,

2. sioctiAsnc INTEGIULS 279

t2, .. .) c [O, co). Let A E PI, s > 0. and /E e(R'). Then A n {r = t ,} E 9,,, so

(1.13) f j ( W ( r + s)) d P A n I? = trl

= f

= I 7 w f ( W ( t , ) ) dP.

E L / ( w, + s)) - W,) + W,)) I Sl,l d P A n lr = ti)

= EEf(& + 4 3 I x = W(tr) dP A n (r = ti)

A n ( I = ( ( )

The verification of (a) is completed as in the proof of Theorem 2.7 of Chapter 4.

Applying the Markov property (with respect to {.Ft)), we have

for allfe 4W') and f > s 2 0, hence for all/€ C(W') with polynomial growth. Taking f ( x ) = xi and then J ( x ) = xixj, we conclude that W; is a continuous, square-integrable, {SP,}-martingale, and

( I . 15) E[Wdr)W'r) I S,] =c Wds)W's) + d,Jt - s), I > s 2 0,

for i, j = I, . . . , d. This implies (b).

provided we can show that, for everyfe Cp(W'), Part (c) follows from Theorems 6.5 of Chapter I and 2.6 of Chapter 4,

(1.16)

as n-+ 00, uniformly in x E W'. Observe, however, that this follows imme- 0 diately from (1.7) and (1.8) if we replace t and 2 by l/n and (, .

2. STOCHASTIC INTEGRALS

Let (0, S, P ) be a complete probability space with a filtration (S,} such that Po contains all P-null sets of 9. Throughout this section, (9,) implicitly prefixes each of the following terms: martingale, progressive, adapted, stopping time, local martingale, and Brownian motion.

Let -U, be the space of continuous, square-integrable martingales M with M ( 0 ) = 0 as. Given M E A,, denote its increasing process (see Chapter 2,

Section 6) by (M), and let L’((M)) be the space of all real-valued, progressive processes A’ such that

E[b2 d ( M ) ] < m, t B 0.

In this section we define the stochastic integral

g;, d M

for each X E L?((M)) as an element of A, itself. Actually, (2.2) is uniquely determined only up to indistinguishability. As in the case with conditional expectations and increasing processes, this indeterminacy is inherent. There- fore we adopt the convention of suppressing the otherwise pervasive phrase “almost surely” whenever it is needed only because of this indeterminacy.

Since the sample paths of M are typically of unbounded variation on every nondegeneratc interval, we cannot in general define (2.2) in a pathwise sense. However, if the sample paths of X are of bounded variation on bounded intervals, then we can define (2.2) pathwise as a Stieltjes integral, and integrating by parts gives

(2.3) d M = X(t )M(t ) - In particular, when X belongs to the space S of real-valued, bounded, adapted, right-continuous step functions, that is, when X is a real-valued, bounded process for which there exist 0 = to < t , < ti < * * - with t,,+ 00 such that

m

2.1 Lemma If M E Yn, and X E S, then (2.5) defines a process fb X dM E A, and

If, in addition, N E ..4Yc and Y E S, then

1. STOCHASTIC INTLCUALS 281

Proof. Clearly, (2.5) is continuous and adapted, and it is square-integrable because X is bounded and M E &,. Fix t z s z 0. We can assume that the partition 0 = to < t , < . - - associated with X as in (2.4) is also associated with Y and that s and t belong to it. Letting

(2.8)

we have

and

(2.10) €[J'b. d M LI' d N - P Y d ( M , N)

- { l X dM [ Y d N - k Y d ( M , N )

= E [ ( p J d M ) ( [ Y a dN) - r X Y J

d ( M , N)19,]

= 0,

where sums over i range over ( i 2 0: t , 2 s, t,+ I s c), and similarly for sums over j. The final equality in (2.10) follows by conditioning the (i, j)th term in the first sum on PI,.,, and the ith term in the second sum on 9,, (as in (2.9)).

0

To define (2.2) more generally, we need the following approximation result.

This gives (2.7) and, as a special case, (2.6).

2.2 Lemma { X n } c S such that

(2.1 1)

If M E A, and X E L 2 ( ( M ) ) , then there exists a sequence

tim .[f(X,, - x)2 d < M ) ] = 0, t 2 0. n-m 0

Proof. By the dominated convergence theorem, (2.1 I ) holds with

(2.12) Xn(c) = X ( ~ ) X [ n, ndX(c))*

which for each n is bounded and progressive.

Thus we can assume that X is bounded. We claim that (2.1 1) then holds by the dominated convergence theorem with

(2.13) XAt) = {(M), - ( W t - , - i A t + n -y X(u) d ( ( M ) , + u), I - a - I A t

which for each n is bounded, adapted, and continuous, Here we use the fact that if h E B[O, m) and p is a positive Bore1 measure on [0, co) without atoms such that 0 < p((s, r ] ) < a0 whenever 0 s s < t c 00, then

(2.14)

Of course, this is well known when p is Lebesgue measure, in which case E is allowed to depend on t . In the general case, it suffices to write the left side of (2.14) as

(2.15)

where F(t ) 5 ~( (0 , t ] ) , and to apply the Lebesgue case.

that (2.1 I ) holds with

h d p = h(t) pa.e. I' ( - # A 1

lim p((t - E A t , 13)- 8-O+

lim (F( t ) - F(t - & A t ) ) - ' i;;*A,w-l(u)) du, r - 0 t

Thus, we can assume that X is bounded and continuous. It then follows

(2.16)

which for each n belongs to S.

X,(t) = x p ) . n

0

The following result defines the stochastic integral (2.2) for each M E

and X E L2( (M)).

2.3 Theorem Let M E ,rV, and X E f?( (M)) . Then there exists a unique (up to indistinguishability) proccss & X dM E dC such that whenever {X,} c S satisfies

(2.1 7)

we have

(2.1 a) O s i s t sup p " d M - p d M ( - r O , r>o,

8,s. and in L2(P) as n-r 00. Moreover, (2.6) holds, and

(2.19) E[(Jlo'X dM)*] = E [ P ' d ( M ) ] .

If, in addition, N E ,rV, and Y E L2(<N)), then

2. STOCHASTIC INTEGRALS 283

for all t 2 0, and (2.7) holds.

Proof. Choose ( X , } c S satisfying (2.17); such a sequence exists by Lemma 2.2. Then, for each T > 0,

by Proposition 2.16 of Chapter 2 and by Lemma 2.1 of this Chapter. In particular, the sum inside the expectation on the left side of (2.21) is finite a.s. for every T > 0, implying that there exists A E F with P(A) = 0 such that, for every w E R, {xAr 1; X , dM) converges uniformly on bounded time intervals as n-, 00. By Lemma 2.1, the limiting process, which we denote by jA X d M , is continuous, square-integrable, and adapted. (Note that A E Po by the assumption on {F,} made at the beginning of this section.) Clearly, (2.18) holds a s

Moreover,

281 STOCHASTIC INTEGRAL EwnoNs

for each T > 0 and each n, so (2.18) holds in t'(P). If {X:} c S also satisfies (2.7) (with X , replaced by XA), then

(2.23)

for each T > 0 and each n. Together with (2.22) this implies the uniqueness (up to indistinguishability) of X d M . To show that & X dM belongs to ,rV, and satisfies (2.7). it is enough to

check that

(2.24a)

and

whenever r 2 s 2 0, where we use the notation (2.8). But these follow immediately from the fact that they hold with X replaced by X , and the fact that (2.18) holds in L?(P).

Suppose, in addition, that N E At and Y E L'((N)), fix r z 0, and let r E a[O, r]. Since (M + aN) = ( M ) + 2a(M, N ) + a'(N),

for all a E R, and hence

From (2.26) and the Schwarz inequality, we readily obtain (2.20) in the case in which X and Y are simple functions (that is, linear combinations of indicator functions). A standard approximation procedure then gives (2.20) in general. To complete the proof, we must check that

(2.27) E[(p a d M ) ( r Y 1 d N ) - f X Y s d < M , N ) l I . ] 5 0

2. STOCHASTIC INtECRALS 285

for all t 2 s 2 0. Let { V, ) c S be chosen by analogy with (2.17). Then by (2.10), (2.27) holds with X and Y replaced by X, and V,. Applying (2.18) (in t z ( P ) ) and its analogue for Y as well as (2.20) with XY replaced by (X, - X)V , and by X( Y, - Y) (which sum to X , U, - X Y), we obtain the desired result by passing to the limit. 0

Before considering further generalizations, we state two simple lemmas. For the first one, given M E dr and a stopping time r, define M' and (M)' by

(2.28) M'(t) = M ( t A r ) , (M): = ( M ) f h r , t 2 0,

and observe that M' E -rY, and

(2.29) (M') = (M)',

2.4 lemma If M E Ac, X E L.(( M)), and r is a stopping time, then

(2.30) L X dM' = l A r X dM, t 2 0.

Proof. Fix t 2 0. Observe first that

(2.31) E c ( s d X dM')'] = E[bz d(M')] = E [ l A * X 2 d(M)]

by Theorem 2.3 and (2.29). Second,

(2.32) .[(LA'. dM>'] = E [ l * ' X 2 d ( M ) ] ,

also by Theorem 2.3. Finally,

(2.33) E [ ( l X d M ' ) ( l A r X dM)] = .[(LA'. dM')(S"X dM)]

= E [ l A r X z d(M', M)] = E [ l A ' X z d(M)].

where the first equality is obtained by conditioning on iF , * ( . the second depends on (2.7), and the third follows from the fact that (M', M ) f , , , =

We conclude that

(2.34) E [ ( P dM'- [*'A' dM>'] = 0.

which sullices for the proof. 0

286 STocNAS7lC W Q A l EQUATIONS

2.5 lemma If M E 4,. X is progressive, Y E L'((M)), and X Y E L?((M)), then X B e((& Y d M ) ) and

(2.35) fi d ( l Y d M ) = [ X Y d M , i 2 0.

Proof. See Problem 11. 0

The integrability assumptions of the preceding results can be removed by extending the definition of the stochastic integral (2.2). Let .&, loc be the space of continuous local martingales M with M(0) = 0 8.9. Given M E Ace l o r ,

denote its increasing process (see Chapter 2, Section 6) by (M), and let L & ( ( M ) ) be the space of all real-valued progressive processes X such that

(2.36) b2 d ( M ) < ca as., t z 0.

If r is a stopping time, define M' and (M) ' by (2.28). and observe that M' E A , 10c and (2.29) holds.

2.6 Theorem Let M E yUc,lOc and X E L&((M)). Then there exists a unique (up to indistinguishability) process fi X dM E .U,, such that whenever r is a stopping time satisfying M' E A, and X E L'((M*)), we have

(2.37) l A ' X dM = dMr, t 2 0.

(The right side of (2.37) is defined in Theorem 2.3.) Moreover, (2.6) holds. If, in addition, N E AC.,- and Y E l&((N)), then (2.20) and (2.7) hold.

Proof. Given M E &c,loc and X 8 L&((M)), there exist stopping times r , 5 72 s with rm-+ 00 such that for each n 2 1, Mrn E 4, and X' d ( M ) s n, implying

(2.38) E[p2 0 d(M')] = E [ l A ' " X 2 d ( M ) ] s n, t 2 0,

and hence X E L?((Mrm)).By Lemma 2.4,

(2.39)

for all m, n 2 I, and existence and uniqueness of I i X dM follow. The conclusions (2.6). (2.20). and (2.7) follow easily from Theorem 2.3 and (2.37). 0

lA1' X dMrm = fi dMeaA'n = l A r m X dM'*, t 2 0,

We need the analogues of Lemmas 2.4 and 2.5. The extended lemmas are immediate consequences of the earlier ones and (2.37).

2. STOCHASTIC INTEGRALS 287

2.7 lemma If M E X E L i c ( ( M ) ) , and t is a stopping time, then (2.30) holds.

2.8 lemma If M E .,4’c.larr X is progressive, Y E L i c ( ( M ) ) , and XY E f&((M)), then X E L.ic((!i Y dM)) and (2.35) holds.

The next result is known as Itd’s formula.

2.9 Theorem For i = I , ..., d, let V, be a real-valued, continuous, adapted process of bounded variation on bounded intervals with YLO) = 0, let M, E A,,,,,, and suppose that X, is a real-valued process such that XAO) is F,-measurable and

(2.40) r z 0.

Put x = (XI , . . ., x d ) and let j ’ ~ C1*2([0, 03) x Rd), that is, J J , I,,, and/;,,, exist and belong to C([O, 00) x Rd) for i , j = I , . . ., d. Then

X,(t) = XAO) + VJr) + MAr),

(2.4 I ) JO, Nr)) - /CO, X(0))

I = I

2.10 Remark form

(2.42)

ltals formula (2.41) is often written in the easy-to-remember

d

df(t, xw) =A09 X ( t ) ) df + c sxj4 XO)) dXXt) I = I

i d

where dXXr) = dVAt) -+ dM,(t) and dXXr) dX,(r) is evaluated using the “multiplication table ’*

(2.43)

dM,( t ) I 0 4 M , , M,), . Proof. Denoting by I

(2.44)

I(r) the total variation of V, on [O, r ] , let

t 2 0: max (IXA0)l + 1 %1(r) -t IM,(t) l ) 2 n 1 6 1 S r l

0

for each n, and note that 7,3 00. Thus it suffices to verify (2.41) with t replaced by t A T, for each n. But this is equivalent (by Lemma 2.7) to proving (2.41) with X I , 5, and M, replaced by Xj", Vj", and Mj" for each n. We conclude therefore that it involves no loss of generality to assume that XAr), I & J ( t ) , M,(t), and (MJI are uniformly bounded in t 2 0, o E Q, and i = 1, ..., d.

With this assumption, we can require that f have compact support. Fix t 2 0, and let 0 = to < t l < < tm = t be a partition of [O, t ] . For the remainder of the proof, we use the notation that for a given process Y (real- or BB'-valued), Ak Y = y(tk+ ,) - y(tk) for k = 0, . . . , m - 1. By Taylor's theorem,

(2.45) S(t, X ( 0 ) -f(4 X(0))

where I <I - x ( t k ) I 5 I X(lk+ I ) - x ( t k ) 1 * The proof now consists of showing that, as the mesh of the partition tends

to zero, the right side of (2.45) converges in probability to the right side of (2.41).

Convergence of the sum of the first two terms in (2.45) to the sum of the first three terms in (2.41) is straightforward (see Problem 12).

Note that by Proposition 3.4 of Chapter 2 and the continuity and bounded variation of the 5, (2.46)

2 STOCHASTIC INTLCRALS 289

the last term i n (2.45) to the last term in (2.41) is a consequence of the following lemma. 0

2.11 Let f be continuous, and let F be nonnegative, nondecreasing, and right continuous on LO, 00). For n = 1, 2, .. ., let 0 = C: < t; < < .... with t;-+ 00. Suppose for each t > 0 that max,,.,, ( t ; + , - t ; ) - * O as n+ m, and suppose thatf, and a, satisfy a, 2 0,

lemma

(2.49) lim a,(t!) = F(c) n-m Ik"$t

for each t at which F is continuous. Then

for each t at which F is continuous.

Proof. Clearly

(2.51) lim [ c h(t:)an(r:) - c ~ ( f ; b n ( ~ ] 0.

Suppose r is a continuity point of F and F ( t ) > 0. Let p, and p be the probabil- i ty measures on [O, r] given by

n + m w s t IL" 5 1

(2.52)

0


We conclude this section by applying 116's formula to give an important characterization of Brownian motion, which is essentially the converse of Theorem 1.2(b).

2.12 Theorem Suppose that X, , ..., X, E A, lor satisfy (X,, X,), = 6,,c for i, j = 1, . .., d and all t 2 0. Then X = (XI, .. . , X,) is a d-dimensional Brown- ian motion.

Proof. Let 8 E w" be arbitrary, and definefi [O, a) x R'-+ C by

(2.54)

where i = tinuous, local martingale, bounded on bounded time intervals, so

(2.55)

f ( t , X) = exp {ie . x + + l O l z t } ,

By Theorem 2.9, {/(& X(t)), t 2 0) is a complex-valued, con-

x(t)) I p.1 =m, X(S))

for all t 2 s 2 0, that is,

(2.56)

Consequently, X is a d-dimensional Brownian motion. E[exp {ie * (X(r) - X(s))> I P,] 5 exp { -41 Ojz ( t - s)}.

0

3. STOCHASTIC INTEGRAL EQUATIONS

Let u: LO, a) x add W d @ IR' (the space of real, d x d matrices) and 6: [0, 00) x R'- R" be locally bounded (i.e., bounded on each compact set) and Bore1 measurable. In this section we consider the stochastic integral equation

(3.1) X( t ) = X(0) + &, X(S)) dW(s) -t b(s, X(S)) ds, f 2 0,

where W is a d-dimensional Brownian motion independent of X(0) and I;, a(s, X(s) ) dW(s) denotes the R'-valued process whose ith component is given by

ld l

Observe that (3.2) is well-defined (and is a continuous, local martingale) if X is a continuous, Rd-valued, (g1}-adapted process, where 9, = PP," V a(X(0)). In the classical approach of It6, W and X(0) are given, and one seeks such a solution X. For our purposes, however, it is convenient to regard W as part of the solution and to allow {FI} to be an arbitrary filtration with respect to which W is a d-dimensional Brownian motion.

3. STOCHASTIC INTEGRAL WuAnoNs 291

Let p E g(IF4'). We say that (Q, 9, P, {9,}, W, X) is a solution of the stochastic integral equation corresponding to (a, b, p) (respectively, (a, b)) if:

(3 (Q, 9, P) is a probability space with a filtration {S,}, and W and X

(b) W is a &dimensional IS,}-Brownian motion. (c) X is {#,)-adapted, where #, denotes the P-completion of 9,. that

is, the smallest a-algebra containing 9, u ( A c Q: A c N for some N E 9 with P ( N ) = 0) .

(d) PX(O)-' = p and X has sample paths in C,,[O, 00) (respectively, X has sample paths in C,JO, 00)).

(el (3.1) holds as.

are UV-valued processes on (a, 9, P).

The definition of the stochastic integral is as in Theorem 2.6, the roles of (Q9, P) and (9,) being played by (Q, #, P) and {#,}. Because it is defined only up to indistinguishability, we continue our convention of suppressing the phrase "almost surely" whenever it is needed only because of this indeterminacy.

There are two types of uniqueness of solutions of (3. I ) that are considered:

Pathwise uniqueness is said to hold for solutions of the stochastic integral equation corresponding to (a, b, p) if, whenever (a, 9, P, {gF,}, W, X) and (Q, 9, P, {R,}, W, X' ) are solutions (with the same probability space, filtration, and Brownian motion), P{X(O) = X'(0) ) = l implies P(X(t) = X'(r) for all r 2 0 ) = 1. Disrribution uniqueness is said to hold for solutions of the stochastic integral equation corresponding to (a, 6, p) if, whenever (a, 9, P, {S,}, W, X) and (a, f', P, {.a";}, W', X') are solutions (not necessarily with the same probability space), we have PX-' = P(X')-' (as elements of @(C,,[O, 00))).

where

(3.4)

and

(3.5) a = aa'.

Observe that if (Q, 9, P, {PI}, W, X ) is a solution of the stochastic integral equation corresponding to (a, b, p), then, by It& formula (Theorem 2.9).


(3.6)

f(x(t)) - I'c/(s* ~ ( s ) ) ds = j ( ~ ( o ) ) + lL,(x(s)bijs, ~ ( s ) ) d ~ , ( s ) 0 1.1-1

for all c 2 0 and f E CT(Rd), so X is a solution of the C,,[O, co) martingale problem for (A, p) with respect to

In what sense is the converse of this result true? We consider first the nondegenera te case.

3.1 Proposition Let u: [0, 00) x Rd-+ Rd@ Rd and b: [0, 00) x Bad+ Wd be locally bounded and Bore1 measurable, and let p E 9(Rd). Suppose u(r, x) is nonsingular for each (t, x) E [0, 00) x Rd and u - * is locally bounded. Define A by (3.3H3.5). If X is a solution of the C,[O, a) martingale problem for (A, p) with respect to a filtration {St} on a probability space (Q, S, P), then there exists a didimensional {#l}-Brownian motion W such that (Q, 9, P, {#,}, W, X) is a solution of the stochastic integral equation corresponding to (a, b, p).

Proof. Since

(3.7)

belongs to Ac for everyfE C:(Wd), it follows easily that (3.7) belongs to uUc,I, for everyfe Cm(R'). In particular,

(3.8)

belongs to A,, and, by (3.6) and Theorem 2.6 (see (2.7)), rt

(3.9)

We claim that

(3.10)

3. STOCHASTIC INl€CRAL EQUATIONS 293

where the second equality depends on Theorem 2.6. Consequently, by Lemma 2.8,

(3.12) ~ Q ( s , X ( S ) ) dW(s) = d M ( t ) = X ( t ) - X ( 0 ) - l 0 for all t 2 0, which is to say that (3.1) holds.

We turn to the general case, in which u may be singular. Here the conclusion of Proposition 3.1 need not hold because X may not have enough ran- domness in terms of which to construct a Brownian motion. However, by suitably enlarging the probability space, we can obtain a solution of (3.1).

It will be convenient to separate out a preliminary result concerning matrix-valued functions.

3.2 Lemma Let o: [O, ac) x R d + Rd 63 Rd be Borel measurable, and put a = UQ? Then there exist Borel measurable functions p, q : [O, 00) x R" -+ Wd @ Rd such that

(3.13) papT + qqT = I d ,

(3.14) uq = 0,

(3.15) ( I , - ap)a(I, - ap)' = 0.

Proof. Suppose first that u i s a constant function. Since a E S, (the set of real, symmetric, nonnegative-definite, d x d matrices), there exists a real, orthogonal matrix U (i.e., U U T = U T U = I,) and a diagonal matrix A with nonnegative entries such that a = U r A U . Moreover, a has a unique &-valued square root a"', and all' = UTA'/'U. Let r be a diagonal matrix with diagonal entries I or 0 depending on whether the corresponding entry of A is positive or 0, and let A , be diagonal with A, A = r. Then, since uu' = U'AU, we have ( A ~ ' Z U ~ ) ( A : / 2 U ~ ) T = r. By the Gram-Schmidt orthogonalization procedure, we can therefore construct a real, orthogonal matrix V such that Ai/2Ua = T V , and hence I-Ua = A ' / ' V . It follows that Uo = A' '*V, from which we conclude u = a'"UrV. It is now easily checked that (3.13H3.15)

To complete the proof, it suffices to show that the measurable selection theorem (Appendix 10) is applicable to the multivalued function taking o E W d @ Rd to the set of pairs (V, V) as above. The details of this step are left to the reader. 0

hold with p = VTA:"U and Q =: v'(Id - t-)U.

3.3 Theorem Let u: [O, ao) x Eld-+ Old@ Wd and h: [O, 00) x R'-+ Rd be locally bounded and Borel measurable, and let p E 9 ( R d ) . Define A by (3.3)- (3.5). and suppose X is a solution of the C,,[O, 00) martingale problem for ( A , p) with respect to a filtration (9,) on a probability space (a, 9, P). Let W' be a &dimensional {9;)-Brownian motion on a probability space (fl', F',

P’) and define 0 = Q x a, # = m, = P x P, #t = m, and W(t , w, w’) = X(t, w). Then there exists a .d-dimensional (#l}-Brownian motion such that (fi, #, p, (.#t}7 m7 x ) is a solution of the stochastic integral equation corresponding to (a, b, p).

Proof. Define M by (3.8), n(t, o, a’) r M(t , w), and tt”(t, 0, a’) = W’(t, a‘). Using the notation of Lemma 3.2, we claim that

(3.16)

= S,,t, t 2 0, where the first equality uses the fact that <Ak, R) = 0 for k, I = 1, ..., d, the second depends on Theorem 2.6, and the fourth on (3.13). By Lemma 2.8, (3.14), and (3.19,

3. STOCHASTIC INTEGRAL EQUATIONS 295

where the next-to-last equality is a consequence of (3.15) and

and the desired result follows. 0

3.4 Corollary Let a: [0, 00) x R'-+ R'@ W' and 6: [O, 00) x UP+ R' be locally bounded and Bore1 measurable, and let y e @a\'). Define A by (3.3)- (3.5). Then there exists a solution of the stochastic integral equation corresponding to (a, b, y) if and only if there exists a solution of the Cw[O, 00) martingale problem for (A, y). Moreover, distribution uniqueness holds for solutions of the stochastic integral equation corresponding to (a, 6, y) if and only if uniqueness holds for solutions of the C,,[O, 00) martingale problem for (A, PI.

3.5 Proposition Suppose, in addition to the hypotheses of Corollary 3.4, that there exists a constant K such that

(3.20) la(t, x)l 5 K(l + 1 ~ 1 ~ ) . x . b(t, x) 5 K(I + I XI'), r 2 0, x E R?

Then every solution of the martingale problem for (A, p) has a modification with sample paths in C,,[O, 00). Consequently, the phrase "CR.[O, a)) martingale problem" in the conclusions of Corollary 3.4 can be replaced by the phrase '' martingale problem."

Proof. Let X be a solution of the martingale problem for (A, p). By Theorem 7.1 of Chapter 4, Xo(t) 3 (t, X(t)) is a solution of the martingale problem for A'. where

(3.21)

By Corollary 3.7 of Chapter 4, Xo has a modification Y o with sample paths in Qlo, m, w)aCO, 00). Letting

(3.22) (Ao)' = { (J g) E C((C0, 00) x W'IA) x B(([O, 00) x R'P):

A0 = {(I!! YGf+ I ! n : f E ccm(m Y c c,"o* 00)).

(! d l l o . m ) w ~ E A', .f@) dA) = 0).

i t follows that Y o is a solution of the martingale problem for (Ao)'. Choose cp E CXO, 00) with xl0. 5 cp 5 xlo. z1 and v' s, 0. Then the sequence

2% STOCHASTIC INTHiW EQUATIONS

U,, gn)) c (A')* given by M, x) = dl/n)rp( I x t2/n'), L(A) = 0, and g,, = (A")Y,, satisfies bp-lim,,,, f . = ~ I O . m o l r ~ , g , 4 0 pointwise, and supl Ilg;II < 00. By Proposition 3.9 of Chapter 4, Y o has almost all sample paths in DIo, m)xw[O, 00). and therefore, by Problem 19 of Chapter 4, in C,, m, ,JO, co). Define q : [O, ao) x Rd+ Wd by q(r, y) = y. Then 0 Yo is a solution of the martingale problem for (A, p) with almost all sample paths in CR,[O, 00), and the first conclusion follows.

The second conclusion is an immediate consequence of this. 0

3.6 Theorem Let 0 : [O, 00) x Rd+ R'@ Iw" and b: [O, 00) x Rd-+ Rd be locally bounded and Bore1 measurable, and let p E 9(Rd). Then pathwise uniqueness of solutions of the stochastic integral equation corresponding to (a, b, p) implies distribution uniqueness.

Proof. Let (Q 9, I", {S,}, W, X) and (Q', F, P, {Pi}, W', X ' ) be two solutions. We apply Lemma 5.15 of Chapter 4 with E = C,[O, co) x w", S, = S2 = Cw[O, 00) x C,[O, co), P I = P(W, X)-', P, = P(W', X ' ) - ' , and Xl(ol , 0 2 ) = X2(01, 02) = (a1, ~ ~ ( 0 ) ) . Letting A = PW-' = P(W')-', we have P(W, X(O))-' = P'(W', X'(O))-' = A x H. We conclude that there exists a probability space (fi, 9, p ) on which are defined CR,[O, 00)- valued random variables @, R, and satisfying p(@ x)-' = P(W, X)-', f iw, p)-' = J"(W', XI)-', and P(B(0) = f ' ( O ) } -- 1. Moreover, for allJ g E B(c d o , X c&), 00))~

(3.23) EPCf(% &lo, m1

= (B, x)IW)Cc(dJo. Let 0 I; s1 s

j = 0, 1, 2, 3. Then s sk 5 s s t l s s t k and fil c B(E) for i = 1, ..., k and

(3.24)

3. STOCHASTIC INTEGML EQUAnONs 297

where the second equality depends on the fact that the two conditional expectations on its left side are functions only of ( P ( . As), x). I t follows that @ is a d-dimensional {#,}-Brownian motion, where

(3.25) 9, = a(W(s), R(s), R'(s): 0 5 s 5 c),

and hence (a, #, p , (#,}, w, 2) and (fi, 9, p, {#,}, $, 2') are solutions of the stochastic integral equation corresponding to (a, 6, p). By pathwise uniqueness, B(R(r) = P ( r ) for all t 2 0} = 1, so P X - ' = PX- 1 = P ( P ) - l = P'(X')- I . Thus distribution uniqueness holds. 0

The next result gives suflicient conditions for pathwise uniqueness of solutions of (3. I).

3.7 Theorem Let a: [O, a) x Rd+ R d @ R' and b : [O, 00) x Rd-t W' be locally bounded and Bore1 measurable. Let U c !Rd be open, let T > 0, and suppose that there exists a constant K such that

Given two solutions (n, 9, P, {S,}, W , X) and (a, 9, P, {S,), W , Y) of the stochastic integral equation corresponding to (0, b), let

(3.27) t = inf { t 2 0: X ( t ) # V or Y ( t ) 6 V).

Then P{ X ( 0 ) = Y(O)} = 1 implies P { X ( t A r) = Y(t A T ) for 0 5 f s T } = I .

298 STOCHASllC INTEGRAL EQUATIONS

Proof. For 0 s t s T,

(3.28) E [ I X ( t A ? ) - Y(tAr)(’]

In particular, if a(t, x ) and b(t, x ) are locally Lipschitz continuous in x , uniformly in t in bounded intervals (i.e., for every bounded open set U c Rd and T > 0, (3.26) holds for some K), then we have pathwise uniqueness. This condition suffices for many applications. However, in some cases, a = uu’ is a smooth function but u is not. In general this causes serious difliculties, but not when d = 1.

3.8 Theorem In the case d = I, Theorem 3.7 is valid with (3.26) replaced by

Noting that and conclude from the dominated convergence theorem that

s supyeR I y l / ( y 2 + for all u E R and E > 0, we let G-+ 0

1 (3.32) E[ I X(r A 7 ) - Y(t A T ) I] S KE [LA‘ I W s ) - Y(s)i ds

S K E [ I X ( S A T ) - Y(sAr) l ] ds

0

L for 0 s t 5 T, and the result again follows from Gronwall’s inequality.

We turn finally to the question of existence of solutions of (3.1). We take two approaches. The first is based on Corollary 3.4 and results in Chapter 4. The second is the classical iteration method.

3.10 Theorem continuous and satisfy

(3.33)

Let 0 : [0, a) x R’- R d @ W‘ and 6: [0, 00) x R‘-+ W‘ be

lu(t, x)12 S K(1 + l~l’), x . b(t ,x) 5 K(I + Ix12),

t 20, x E R’,

for some constant K. and let p E P(W’). Then there exists a solution of the stochastic integral equation corresponding to (a, 6, p).

Proof. It suffices by Corollary 3.4 and Proposition 3.5 to prove the existence of a solution of the martingale problem for (A,p) , where A is defined by (3.3H3.5). By Theorem 7.1 of Chapter 4 it suffices to prove the existence of a solution of the martingale problem for (Ao, 6’ x p), where A’ is defined by (3.21). Noting that A’ c e([O, 00) x R’) x e([O, 00) x Fa‘) and A’ satisfies the positive maximum principle, Theorem 5.4 of Chapter 4 guarantees a solution of the I+,’, o o ~ x R l ~ b [ O , 00) martingale problem for ((A’)’, So x p), where (Ao)’ is defined by (3.22). Arguing as in the proof of Proposition 3.5, we complcte the

0 proof using Proposition 3.9 and Problem 19. both of Chapter 4.

300 STOCHASTIC INTEGIAL EQUAllOhlS

3.11 Theorem Let 0 : 10, 00) x Rd-t Rd@ Rd and b: [O, 00) x Rd-+ Rd be locally bounded and Borel measurable. Suppose that for each T > 0 and n z 1 there exist constants h'r and KT, such that

(3.35) l a x) - 4 4 Y)l v IW, x) - b(4 Y)l 5 K , , I x - Y l , 1x1 v I yl I; tl. 0 5 f s T ,

Given a d-dimensional Brownian motion W and an independent W-valued random variable c' on a probability space (0, S, P) such that E [ I (1'1 < ao, there exists a process X with X(0) = c' as. such that (Q 5, P, {SJ, W, A') is a solution of the stochastic integral equation corresponding to (a, b), where .FI = 9: v t7(<).

Proof. We first give the proof in the case that (3.34) and (3.35) are replaced by

and

and note that E[IXk, , ( r ) l2J < 00 for each t 2 0 by(3.36). Fork = 0, 1, ..., let cpk(t) = E [ I X I + - Xk(t) 1'1. Given T > 0, (3.37) implies that

Since cp,(t) I; 2K31 + T)t for 0 I; t s T by (3.36), we have by induction that, fork = 0, 1, ...,

[2K+(1 + T ) ] k + ' P + ' , O s r s T . pk(t) (k + I ) !

(3.40)

3. STOCHASTIC INTEGRAL EQUATION5 301

and therefore m

(3.42) f p { sup lxk+ l ( t ) - xk(t)I 2 2-’ 5 4k&k(T) < 00. k = O O a r s T 1 k = O

By the Borel-Cantelli lemma, SUPoscsr lXk+,(t) - xk(t)I < 2-’ for all k 2 &(a) for almost all w. Now T was arbitrary, so there exists A E 9 with P(A) = 0 such that, for every w E R, {zAr xk} converges uniformly on bounded time intervals. Letting X be the limiting process, we conclude from (3.42) that X(0) = 4 a.s. and (R, 9, P, {gCI}, W, X) is a solution of the stochastic integral equation corresponding to (a, b).

We now want to obtain the conclusion of the theorem under the original hypotheses ((3.34) and (3.35) instead of (3.36) and (3.37)). For each n z 1 , define p,: [O, 00) x R’- [0, 00) x R‘ by p,(t, x) = (t, ( I A(n/l xl))x), and let a, = u 0 p , and b, = b 0 p,,. By the first part of the proof there exists a solution (0,s. P, {PI}, W, X,) of the stochastic integral equation corresponding to (a,, b,). Letting T, = inf { t 2 0: IX,,(t)l 2 n } , Theorem 3.7 guarantees that X,(t) = XJt) whenever 0 s t s T , A T,,, and m, n 2 1. Thus, we can define X(t) = X,(t) for 0 s t 5 T , , n 2 1. To complete the proof, i t sufiices to show that r,+ 00 as. By It6’s formula

and (3.34),

is bounded above in n for fixed t 2 0. The same is therefore true of log ( I + nZ)P{r, S t}, so PIT, t } -+ 0 for each t 2 0. Since r I 5 1’. 9 a , the desired conclusion follows. 0

(3.43) EUog (1 + I Xn(t A r n ) 12)1

302 STOCHASTIC INTEGUAL EQUATIONS

4. PROBLEMS

1. Let W be a d-dimensional {.F;,f-Brownian motion, and let T be an {.F,}-stopping time with ‘c -z og as. Show that W * ( . ) 3 W(r + a ) - W(T) (E 0 if T = ao) is a &dimensional Brownian motion, and that S,“. is independent of .Fc for each i 2 0.

2. Let W be a d-dimensional (9J-Brownian motion. Show that

is an (4F,}-martingale. For d = I , a > 0, and f i > 0, show that P{suposas, ( W(s) - as/2) > p} 5; e -a@.

3, Let W be a one-dimensional Brownian motion. Evaluate the stochastic integral &W2 dW directly from its definition (Theorem 2.3). Check your result using IWs formula.

4. Let M E dc, and X, Y, XI, X 2 , . . . E LL((A4)). Suppose that I X,l s Y for each n z I and X,(t)--+ X( t ) a.s. for each t 2 0. Show that for every T > 0,

(4.2) Q S t S f

5. Let W be a ddimensional (9,}-Brownian motion (with ($3 a complete filtration), and let 6: [O, a) x fl-, R‘@ w‘ be {.F,}-progressive and satisfy onr = 1,. Show that w = jA 4 s ) dW(s) is a d-dimensional (9,)-Brownian motion.

6. Show that the spherical coordinates p 5 IBl = (B: + Bf + Bj)”Z,

(4.3) cp = cos-* (B3/p) = colatitude, 0 = tan-’ (B2 /Bl ) = longitude

of a three-dimensional Brownian motion B = ( B , , B, , B,) evolve according to the stochastic differential equations

dp = dW, f p - ’ dt, dq = p - l dW2 + *p-’ cot cp dt,

d9 = p - ’ csc cp dW3

(4.4)

with a new three-dimensional Brownian motion W = (W, , W,, W,):

W, = l p ' - l ( E 1 d E , + E , dB , + B, dE3) ,

W, = Lp-'(csc cp)E,(E, d E , + E , dE, ) - sin cp d E , ,

W, = L p - ' (csc rp)(E, dB, - 8, dE,) .

(4.5) s.' 7. Let a: [O, CQ) x Rd-+ Rd@ R' and b: [O, 00) x Rd+ R' be locally

bounded and Borel measurable and suppose that (Q, F, P, {PI}, W , X ) is a solution of the stochastic integral equation corresponding to (a, b). Let c: [O. 00) x W'-+ W be bounded and Borel measurable. Show that if

JE C,l* ,([O, a) x W'), then

for all r 2 0, where G is defined by (3.4) and (3.5).

8. Let U): I w ' 4 Rd be a C'-difTeomorphism (that is, U) is one-to-one, onto, and twice continuously differentiable, as is its inverse U)- I). Let 0 : Rd-+ Iw' @I R' and b: Rd-r R' be locally bounded and Borel measurable, and suppose the stochastic integral equation corresponding to (a, b) has a solution (n, 9, P, {.F,}, W , X) . Observe that then there exist 8: Rd-+ R'@ W' and 6: Wd-+ W" locally bounded and Borel measurable such that (Q, 9, P, {.FI), W , @ 0 X ) is a solution of the stochastic integral equation corresponding to (8, 6). Define G in terms of a and b and G in terms of ii and 6 as in (3.4) and (3.5). Show that GJ== [e ( J o U) for all J E Cf(R'). Thus the relationship between a stochastic integral equation and its associated differential operator is invariant under diffeo- morphism.

9. Let a: Rd-+ S d and b : R"-+ R' be locally bounded and Borel measurable, and define A and G by (3.3) and (3.4). Let cp E C2(W') and suppose that for each n 2 1 there exists a constant K, r: 0 such that

(4.7) max {VP * ~ V V , G V I X , ~ s W: dr)>o, 1x1 snl Ka CP-

Show that if X is a solution of the C,,[O, 00) martingale problem for A, then P { dX(0)) s 0 ) = 1 implies P{ cp(X(t)) s 0 for all r 2 0 ) = I . Hint: Show that Gronwall's inequality applies to E[cp+(X(t A T,))], where q, = inf { t 2 0: I X(r) I 2 n}, by approximating cp + by a sequence of the form {h, 0 9).

10. Define &, a@, X(s)) dW(s) for 6: LO, a) x R‘4 Rd @I W” (the space of real d x m matrices) locally bounded and Bore1 measurable with W and X as before by defining 6: [0, 00) x R’vm-+ Rdvm@ Rdvn in terms of o in the obvious way. Check to see which of the results of Section 3 extend to nonsquare a.

11. (a) Let M E -K, and X E L2((M)), and let s 2 0 and 2 be a bounded .F,-measurable random variable. Show that

(4.8) dM = Z X dM, t > s. I‘ (b) Prove Lemma 2.5. Hint: First consider X E S.

12. Let M E ..rY,,r,, and let X be continuous and adapted. Show that for 0 = to < t , < * ‘ . < t, = 1,

(4.9) 6‘. d~ = Jim c X(r,)(M(t, + ,) - M(i,).

13. Let W be a one-dimensional Brownian motion, and let X( t ) = W(t) + t . Find a function cp such that cp(X(t)) is a martingale. (Use 116’s formula.) Let T = inf { t : X ( t ) = - a or 6) . Use cp(X(t)) to find P { X ( T ) = b) . What is E[T] ?

mar (a + I - hj-0 k

14. Let X be a solution in of

X ( t ) = x + bX@) ds + ~ X ( S ) dW(s) S L and let Y = X’. (a) Use Itd’s formula to find the stochastic integral equation satisfied by

Y. (b) Use the equation in (a) to find E[X2]. (c) Extend the above argument to find E [ X k J , k = I , 2,3,. . . ,

15. Let W be a one-dimensional Brownian motion. (a) Let X = (X, , X,) satisfy

X , ( t ) = XI +

X,(f) = x2 - X,(s) ds + cX,(s) dW(S).

XJS) ds ib l l

Define ml(t) = E[X’(t)] , m,(f) = E[X(t)Y(t)] , and ms(t) = E[Y2( t ) l . Find a system of three linear differential equations satisfied by m, ,

5. NOTES 305

m , , and m s . Show that the expected “total energy” ( E [ X 2 ( t ) + Y2(r ) ] ) is asymptotic to kear for some A > 0 and k =- 0.

(b) Let X = (XI, X,) satisfy

X , ( t ) = XAO) + S’x,(s) d W s ) 0

x20) = X 2 ( 0 ) - [X1b) d W ) .

Show that X : ( t ) + X j ( t ) = ( X i ( 0 ) + Xi(0 ) ) el.

16. Let W be a one-dimensional Brownian motion. (a) For x 2 0, let X(t , x) = x + (b AX(s, x) ds + yo ,/m dW(s) and

T , = inf { t : X(r, x) = O}. Calculate f ( r , < a} as a function of A.

(b) For x > 0, let X ( f , x) = x - ro AX(.$, x) ds + yo X(s, x) dW(s) with 1. > 0, and let T , be defined as above. Show that f ( T x < 0 0 ) = 0, but that f{lim,-.m X ( t , x) = 0) = 1.

(c) For x > 0, let X ( t , x) = x + Po a(X(s, x)) dW(s), and let T, be defined as above. Give conditions on CT that imply E[?J < 00.

(d) For x > 0, let X(t, x) = x + ro A ds + Yo ,/= dW(s), and let T~ be defined as above. For what values of 1 > 0 is P(T, < a} z O? For these values, show that P { T ~ < 00) = 1, but that E[T, ] = 00.

5. NOTES

There are many general references on stochastic integration and stochastic integral equations. These include McKean (1969), Gihman and Skorohod (1972), Friedman (1975). Ikeda and Watanabe (1981). Elliot (1982), MCtivier (1982), and Chung and Williams (1983). Our treatment is heavily influenced by Priouret ( I 974).

Stochastic integrals with respect to square integrable martingales go back to Doob (1953), page 437, and were developed by Courrege (1963) and Kunita and Watanabe (1967). The extension to local martingales is due to Meyer (1967) and Doleans-Dade and Meyer (1970). It8’s formula goes back, of course, to I t6 (1951).

Theorem 3.3 is due to Stroock and Varadhan (1972), Theorems 3.6 and 3.8 to Yamada and Watanabe (1971), and Theorem 3.10 to Skorohod (1965). Theorems 3.7 and 3.1 I are the classical uniqueness and existence theorems of It6 (1951).

Problems 6 and 8 were borrowed from McKean (1969) and Friedman (1975). respectively.

In this chapter we continue the study of stochastic equations that determine Markov processes. These equations involve random time changes of other Markov processes and frequently reduce stochastic problems to problems in analysis. Section 1 considers random time changes of a single process. The multiparameter analogue is developed in Section 2. Section 3 gives convergence results based on the random time changes. Sections 4 and 5 give time change equations for large classes of Markov chains and diffusion processes.

1. ONE-PARAMETER RANDOM TIME CHANGES

Let Y be a process with sample paths in Da[O, a), and let j? be a nonnegative Borel measurable function on E. Suppose that p 0 Y is as. bounded on bounded time intervals. We are interested in solutions of



1. ONE-PARAMETER RANDOM TIM€ CHANGES 307

then (see Problem 1 I )

and equality holds if and only if the Lebesgue measure of (s s r : /1(Z(s)) = 0) is zero. Conversely if

(1.4)

has a solution for all r (of necessity unique), then r ( r ) is locally absolutely continuous (in fact locally Lipschitz) with

(1.5) W) = P(Y(dt))) , a.e. t,

(differentiate both sides of (l.4)), and hence Z(r) = Y(r( t ) ) is a solution of (1.1). More generally, let

" I r l = inf {s: du = 00

and suppose r(t) satisfy (1.4) for

(]ID( Y(u))) du = 00. If r 1; r , and /?( Y(r) ) = 0 when r < a), let

and

(1 -8) r(r) = r, r > to.

Then Z(r) ZE Y(r(t)) is a solution of (1.1).

1.1 Thcorem

(1.9)

Let Y, fl, and r l be as above. Define

to = inf (s: /?( Y(s)) = 0)

and

(1.10) r2 = lim inf (s: @(Y(s)) < E ) . e - 0 t

(a) If ro = r , and fl(Y(rO)) = 0 when to < 00, then (1.1) has a unique solution Z(t).

308 RANDOM TlML CHANCES

(b) Suppose /I is continuous. If to = T, = T ~ , then there is a unique locally absolutely continuous function y(r) satisfying

1.2 Remark Note that t o , T~ , and 72 may be infinite. 0

Proof. Existence follows from the construction in (1.7) and (1.8). By (1.3), any solution, Z(t), with 7( t ) defined by (4.2), must satisfy f(t) s t I . If T(r) < T ~ , then /?(Z(s)) # 0 for all s 5 t, and (1.4) uniquely determines r(t). If 7, = to, r(t) is uniquely determined for all t .

If r(t) satisfies (1 .1 I), then as above

(1.12)

since Y(u) = Y - ( u ) for almost every u, and y( t ) 5 t l . Since r2 = T, , fl(Y(u))A p( Y -(u)) # 0 for u < 7, , and hence for y(t ) < tl ,

and (1.12) and (1.13) imply Z(y(r)) is the unique solution of(l.1). U

We now relate solutions of ( 1 . 1 ) t o solutions of a martingale problem.

1.3 Theorem Let A c C(€) x B(E) and suppose Y is a solution of the D,[O, 00) martingale problem for A.

If the conditions of Theorem l.l(a) hold for almost every sample path, then the solution of (1.1) is a solution of the martingale problem for P A n (B(E) x B(E)), where

1. ONE-PARAMETER R A N D O M TIME CHANGES 309

Proof. Note that { d s ) I r } = {yo ( l / P ( Y ( u ) ) ) du 2 s} u { T , I I ) E .aFr+, so T(S)

is an {F:t )-stopping time. For ( J g) E A

(1.15)

is an (FQrr,,,+}-martingale. 0

We have the following converse to Theorem 1.3.

1.4 Theorem Let ( E , r) be complete and separable, and let A c C ( E ) x B(E). Suppose 9 ( A ) is separating, the DEIO, a) martingale problem for A is well- posed, and /3 E M(E), @ 2 0, is such that BA c B(E) x B(E). If Z is a solution of the D,[O, 00) martingale problem for PA, then there is a version of Z satisfying ( 1 . 1 ) for a process Y that is a solution of the martingale problem for A.

Proof. First suppose

(1.17) ?(a) E J p(Z(s)) ds = a, a s 0

and define

(1.18) y ( t ) = inf { u : lP(Z(s) ) ds > r } .

Then Y(r) = Z(y(r)) is a solution of the martingale problem for A, that is, for (5 B) E 4

is a martingale by the optional sampling theorem. (See Problem I2 for the equality in (1.19)) Let y t ( t ) = limS-.,+ y(s). We claim that Z is constant on the interval [YO), y + ( t ) ] (see Problem 45 of Chapter 4), and since y(r( t ) ) I; r 5;

Y +(?to)* (1.20)

If i(o0) < m, then Y ( t ) = Z(v(r)) for r I, ~ ( o 0 ) and Y must be extended past 0 T(CO) (on an enlarged sample space using Lemma 5.16 of Chapter 4).

310 RANDOM TlMf CHANCES

1.5 Theonm Suppose Y is as above, is continuous, { V.} is a sequence of processes with sample paths in &[O, ao) such that 5- Y, and (p,) is a sequence of nonnegative, Bore1 measurable functions on E such that

(1.21) lim sup I /3,(x) - B(x) I = 0 n-(0 x a K

for every compact K. Let h, > 0 and limm...m h, = 0. Suppose Z, Z,,, and W,, satisfy

(1.22)

(1.23)

and

(1.24)

for all t 2 0. If 70 = rl = 72 as., then 2, - Z and W, * Z.

1.6 Remark If Y is quasi-left continuous (see Theorem 3.12 of Chapter 4), in particular if Y is a Feller process, then T~ = T~ as. Observe that, for E > 0, 7(') = inf {s: /?(Y(s)) s e} s f o r and by quasi-left continuity, Y(72) = lirn,,o Y(T"') on the set {t2 < a}. Consequently, on {t2 < 00). l i r n ~ ~ o /?( Y(r@')) = /I( Y(72)) -- 0 as., and hence r2 = to. Of course if inf, @(x) > 0, then f o = T, = T~ = m.

Proof. By Theorem 1.8 of Chapter 3 we can, without loss of generality, assume V , and Y are defined on the same sample space (0, S, P ) and that

d( V , , Y) = 0 a.s. Fix o E R for which Iirnmdm d(Y,, Y) = 0 and 70 = T, = r2. We first assume p 3 supx,, &(x) is finite. Then

(1.25)

( I .26)

(1 -27)

and

( I .28) { w(s) : s s; T ) c { Y,(u): u S TB}.

Since { U.) is convergent in DEIO, a), it follows that {Z,,} and (W.} are relatively compact in D,[O, 00). We show that any convergent subsequence of {Z,} (or { W,)) converges to 2, and hence d(W,,

W'(Z . 9 6, T ) 5 w'( u, 9 689 TB),

{Z,(s): s s T) c { Y,(u): u s TP}.

WYK 9 6, T ) w" u, 9 (6 + U P , TB),

d(Z,, 2) = 0 (and

2. MULTIPARAMETER RANDOM TIME CHANGES 311

2) = 0). If lirnk+w d(Z,,, 2) = 0, then

( I .29)

and

lim ~p,cZ.,C.)) ds = l/l(z(s)) ds = y( t ) h - m 0

( I .30)

for almost every t . The right side is either /?( Y(y(r))) or /?( Y - (y(t))) . Therefore y ( r ) satisfies (1.1 I ) and hence 2 = 2. Similarly, if limk-m d(W,, , 2) = 0, then

(1.31) k - m JO Jo

and the proof follows as for {Z , , ] . Now dropping the assumption that the & are bounded, let ZM, Zf, and

Wf be as above, but with /? and /I,, replaced by MA and M A/?,,, M > 0. Since we are not assuming the solution of (1.23) is unique, take Zf = Z,(Of(t)) where

(1.32)

As above, fix w E f l such that d ( 5 , Y) = 0 and to = T , = t 2 . Then limn-.m d(ZF, ZM) = 0. and d(W:, 2'") = 0. Fix t > 0, and let M > supss, @(Z(s)). Note that ZM(s ) = Z(s) for s 5 t . We claim that for n sufficiently large, M > supsS, jI,,(Z~(s)) and hence Zf(s) = Z,,(s) for s s 1. (Similarly for W.".) To see this, suppose not. Then there exist 0 S s, 9 t (which we can assume satisfy s,,+ so) such that lir~~+~ /~.(Z:(S,,)) L M. But {Z,M(s,,)} is relatively compact with limit points in (ZM(so-), ZM(so)} = (Z(so-), Z(s,)}. Con- sequen ti y,

( I .33) lirn /?,(Zf(s,)) = h P ( Z ~ ( s , ) ) s /?(Z(so - )) V P(Z(so)) < M. - n-o3 n-+m

Recall that if ZM(s) = Z(s) for s I t , then d ( Z M , Z) 5 e-'. Since t is arbi- 0 trary, it follows that d(Z,, 2)- 0.

2. MULTIPARAMETER RANDOM TIME CHANGES

We now consider a system of random time changes analogous to (1.1). For k = 1, 2, . , . , let ( E k , rk) be a complete, separable metric space, and let V, be a process with sample paths in D,,[O, m) defined on a complete probability

space (n, 9, f). Let /?k: n, El-+ [O, 00) be nonnegative Borel measurable functions. We are interested in solutions of the system

where 2 = (2, , 2, , . . .). (SimiIariy we set Y J (Y, , G, . . .).) We begin with the following technical lemma.

2.1 Lemma If for almost every o E n a solution 2 of (2.1) exists and is unique, then Z is a stochastic process.

Proof. Let S = n, D,,[O, ao) and define y: S x S-r S by

Then yk is Borel measurable and hence

(2.3)

is a Borel measurable subset of S x S, as is

r - {(Y, 4: z = rcv, 2))

(2.4) r k , l , B = ((Us 2): = ?(Ys z), zk(f ) E B,

for B E A?(&). Therefore nrk, ,, = ( y : (y, z) E rk, ,, b} is an analytic subset of S and

(2.5) ( z k ( r ) E B, = { nr&. I , B } E

by the completeness of (a, .F, P). (See Appendix 11.) 0

In the one-dimensional case we noted that r(r) was a stopping time with

To determine the analogue of this observation in the multiparameter case respect to 9:+, at least in the case T,, = T , , @(Y(t0)) = Ofor f 0 c 00.

we define

(2.6) p: = sk uk), E lo, O01",

and

where N c gC is the collection of all sets of probability zero, and uc) is defined by up' = uk -k l/n, k s n, and up' = 00, k > n. A random variable r = ( t , , t2, . . .) with values in [O, 00)" is an {#&stopping time if (t s u) = {tl S u1 , t2 s u2, . ..) E S,, for all u E [O, a)". (See Chapter 2, Section 8, for details concerning multiparameter stopping times.)

1. MULTIPARAMETER R A N D O M TIME CHANGES 313

2.2 Theorem (a) For u E [O, mIm, t =- 0, let H,,, be the set of o E R such that there exists z E S 3 n, D,[O, 00) satisfying

and

(2.9) l p k ( z ( s ) ) ds 5 u,, , k = I , 2, . . . .

Then H,, E 9,. (b) Suppose a solution of (2.1) exists and is unique in the sense that for

each t > 0 and almost every w E R, i f z' and zz satisfy (2.8), then z'(r) = zZ(r) , r 5 t. Then for al l t 2 0, r (r ) E (T , ( r ) , rz ( t ) , . . .), with

(2.10)

i s an {.F,}-stopping time.

Proof. (3 Proceeding as in the proof of Lemma 2. I , let I-,, I c S x S be the set of ( y , z ) such that zk(r) = yk(& flk(z(s)) ds), r s t , and (2.9) i s satisfied. Then

H , , , = { Y E nr,,,) E .F,.

(b) By the uniqueness assumption P({r(r ) I u ) A ?fUJ = 0, and hence 0 by the completeness of 9,, { ~ ( t ) 5 u } E 9,.

2.3 Remark If we drop the assumption of uniqueness, then there will in general (even in the one-dimensional case) be solutions for which ? ( I ) is not an

0 {9,}-stopping time. See Problem 1.

Given Y = (Y, , Y z , . . .) on (0, .F, P), we say (2.1) has a weak solution if there exists a probability space (0, ,#, p) on which are defined stochastic processes P = (P,, PZ, .. .) and 2 = (z,, 2,. . ..) such that pis a version of Y and

(2. I I )

2.4 Proposition (2.1) has a solution.

I f (2.1) has a weak solution, then for almost every w E R

314 RANDOM TlME CHANCES

Proof. As in the proof of Lemma 2.1, let S = n k DB.[O, m) and let r c S x S be given by (2.3). Let 9 be as above, and let nT = { y : (y, z) E T}. Then

(2.12) f { Y E d - ) = B { P e n r ) = i ,

that is, (2.1) has a solution for almost every w c n. 0

2.5 Remark In general it may not be possible to define a version of 2 on For example, let f l consist of a single point, let Y( t ) = t, and let Let 4, defined on (h, #, p), be uniformly distributed on 10, 17 and

define

(2.13)

Then for p(r) = t,

(2.14)

but a version of 2 cannot be defined on (n, 9, P). 0

The condition that t ( r ) is in some sense a stopping time plays an important role as we examine the relationship between random time changes and corresponding martingale problems. With this in mind, we say that a stochastic process 2 defined on (n, 9, P) and satisfying (2.1) is a nonanriciparing solution of (2.1) if there exists a filtration {g,} indexed by u E [0, a)" such that F, c g, c 9 (9, given by (2.7)).

(2.15) P{(Y,(U, + a), Y~(u, + *), ...) E Big"}

= P{(Y,(u, + Y,(u, + .), ...) E SlS,)

for all Bore1 subsets B of nk &[O, a), and if t(t), given by (2.10), is a {gu}-stopping time for each t 2 0.

We have three notions of solution, and hence three different notions of uniqueness. We say that strong uniqueness holds if for almost every w E n, (2.1) has at most one solution; we say that weak uniqueness holds if any two weak solutions have the same finite-dimensional distributions; and we say that we have weak uniqueness for nonanticipating solutions if any two weak, nonanticipating solutions have the same finite-dimensional distributions.

We turn now to the analogue of Theorem 1.3. Let &, k = 1. 2, ..., be independent Markov processes corresponding to semigroups { 7&)}. Suppose (&(t)} is strongly continuous on a closed subspace Lk c if&), and let Ak be the (strong) generator for {&( r ) } . We assume that is separating, contains the constants, and is an algebra, and that the D&[O, 00) martingale problem for A is well-posed. By analogy with the one-dimensional case, a solution of (2.1) should be a solution of the martingale problem for

z. MULITPARAMETER RANDOM TIME CHANCES 315

2.6 lemma Let Y, , Y,, .. . be independent Markov processes (as above) defined on (a, 9, P). Then a stochastic process Z satisfying (2.1) is a nonanticipating solution if and only if for every r 2 0, r(r) is a (Y,]-stopping time for some {g,,} satisfying

(2.17) [ 11 j k ( %(Uk + Ok)) 1 .u] = PI &(uk)-fk( yk(uk)) kcI

for all finite I c { I , 2, ...I,/; E f+ , and uk, Uk 2 0, or, setting H k A = Akf,/S,,

(2.18) E n h( h(uk -k 01)) exp { - ' " f j t j.( %(S)) d r } 1 9.1 [ k e I

= n fk(K(uk)) kel

for all finite 1 c { I , 2, ...}, h E ! 2 + ( A k ) , and uk, uk 2 0. ( g + ( ~ , ) =

g(Ak) : f(x) ' Proof. The equivalence of (2.1 7) and (2.15) follows from the Markov property and the independence of the V,. The equivalence of (2.18) and (2.15) follows from the uniqueness for the martingale problem for A, and the independence of the &. If 9, does not contain 9,. then Y,V.Fu still satisfies (2.17) and (2.18). In particular, gu can be replaced by g,, wherc $, is obtained from Yu as

0 9, is obtained from 9:. See (2.7).

2.7 Lemma Let Y, , U, , . . . be independent Markov processes (as above). A stochastic process Z satisfying (2.1) is a nonanticipating solution if and only if

Proof. The necessity of (2.19) and (2.20) is immediate from Lemma 2.6. Define

(2.21) g:., = o(((sk): sk < u k , k 6 f ) V o min(uk - T k ( t ) ) V O : t 2 0). ( k . 1

Then (2.19) implies

E n h( & ( u k + u k ) ) 3:. I = n G ( u k ) . h ( & ( u k ) ) * [ k s I I 1 k r l

(2.22)

Fix I. If I' 3 I , then, by taking$, E 1 for k e I' - I, we can replace g,",l in (2.22) by 9,". I . . If uk = 00 for k $1, then, for I' 2 I , minks I . (uk - f k ( t ) ) V 0 = mink., (uk - t k ( t ) ) V 0, and hence Yt, I . , with I' 3 I, is increasing in I'. For u satisfying u k = 00, k $ 1, we define

(2.23) y," = v Y,qI., 1831

and we note that we can replace Y:, I in (2.22) by 9,". For arbitrary u define

(2.24)

where u t ) = u k + l/n for k 5 n, u k = Q, for k > n. If I c { 1, 2, . . . , n}, we have

The right continuity of K , the continuity of& and 7 i ( u k ) f k s and the fact that YfI., is decreasing in n imply (2.17). A similar argument shows that (2.20) implies (2.18). Finally (r(t) 5 u} = nl n,, {mink I (up) - rk(r)) > 0) E 9". D

2.8 Theorem Let &, k = 1, 2, . . ., be independent Markov processes (as above). Let / 3 k , k = 1, 2, . . . be nonnegative bounded Bore1 functions on E = n, E,, and let A be given by (2.16).

(a) If Z is a nonanticipating solution of (2.1). then 2 is a solution of the

(b) If 2 is a solution of the DJO, 00) martingale problem for A, then martingale problem for A.

there is a version of 2 that is a (weak) nonanticipating solution of (2.1).

2.9 Remark (a) If inf, &(I) > 0, k = 1, 2, . . . , in (b), then Z itself is a nonanticipating solution of (2.1).

(b) The hypothesis that sup, A ( z ) < 00 is used to ensure that A c e ( E ) x B(E). There are two approaches toward eliminating this hypothesis. One

2. MUUIPARAMETER RANDOM T I M CHANGES 317

would be to restrict the domain of A so that A t C(E) x W E ) . (See Prob- lems 2, 3.) The other would be to develop the notion of a "local-martingale problem." (See Chapter 4, Section 7, and Proposition 2.10.)

By (2.17), for fixed w, yt'(uk) = y#(uk + uk), k = I, 2, .. ., are Markov processes corresponding lo Ah and are conditionally independent given 9,, . Therefore

(2.27) E [ M ( u + u) I S,]

= W u ) ,

and hence M(u) is a {9u)-marlingale. By assumption, T ( C ) is a {Y,)-stopping time and the optional sampling

theorem (Theorem 8.7 of Chapter 2) implies M ( T ( ~ ) ) is a Y,,,,-martingale. BUI

(2.28) M(T(t)) = fl h(Zk(l)) exp { - r p k ( Z ( . ) ) H & f & ( z k ( s ) ) d s ] k s I 0

and

(2.29)

is a martingale by Lemma 3.2 of Chapter 4.

(b) The basic idea of the proof is to let

(2.30)

and define

(2.31) &(u) = z&(yk(u)). Note that then, as in the one-dimensional case, the fact that g ( A k ) i s separating implies Z satisfies (2. I).

318 RANDOM TlME CHANGES

Two difficulties arise. First, n ( u ) need not be defined for all u 2 0, and second. even if y k is defined for all u 2 0, it is not immediately clear that the & are independent. Note that for each f > 0 and ( A g) E A,,

Y d U ) A I

(2.32) M(u) = / ( z k ( Y k ( u ) A t ) ) - f lk (z (s) )dzk(s) ) ds

U A I,())

=/( yk(u A r k ( r ) ) ) - J . dY,(s)) ds

is an (.F~,,,,)-martingale. By Lemma 5.16 of Chapter 4, there exists a solution R, l of the martingale problem for Al, and a nonnegative random variable q k ( t ) such that V,( A f&)) has the same distribution as 5, A q k ( t ) ) .

Letting I-+ a, (G,,, q k ( t ) ) converges in distribution in O,[O, a) x [O, a] (at least through a sequence of ts) to (t, m , q,,(ao)) and %(a A r,(a)) has the same distribution as t, ,(. A q k ( a ) ) . In particular,

(2.33) Zk(oo) E lim &(t) = lim K(uA r k ( 0 3 ) ) 1-m u-m

exists O n { T k ( a ) < a}. Fix y k E EI, and Set z k ( a ) = yk On {?,(a) = 00).

Let Q' = 0 x H k DE,[O, a) and define

(2.34) Q(C x B1 x B2 x B , x a * * )

P

for C E 9F and Bk E L3(DE,[0, GO)), where P''' is the distribution of the Markov process with generator A, starting from y. Then Q extends to a measure on F x n,, @(D,[O, a)).

Defining Z on 42' by Z(r, (0, o,, a2, ...)) ZE Z(r, a), we see that 2 on (a, f x f l k W(D,[O, a)), Q) is a version of 2 on (n, 9, P). Let w k denote the coordinate process in D , [ O , 00). that is,

(2.35) wk(t, 0 1 9 0 2 0 3 , * * .)) = o k ( t ) *

Set

(2.36)

allowing t = 00, and define

(2.37)

We must show that there is a family of a-algebras {Y,,) such that T(t) is a {g,,)-stopping time and the U, satisfy (2.17).

(2.39)

and set K h k = (d/dt)hJh, . Setting

Lemmas 3.2 and 3.4 of Chapter 4 imply that

(2.41) n M k ( t ) k c l

is a martingale for any finite I c ( I , 2, ...}, with respect to {Y f } . Defining y k ( u ) by (2.30) for u < ?,(GO) and setting yr(u) = 00 for u L f k ( a ) , Problem 24 of Chapter 2 implies

(2.42)

for u s u, where yI(u) = / \ k c I Yk(Uk). In particular, from the definition of hk

and q k 9

we see that we can drop the '' A t,(oo)"on both sides of (2.43). Let qik(xk) c L, satisfy 0 < ip,, S I, and let 0 s slk s u,. Let p 2 0 be

continuously differentiable with compact support in (0, 00) and p(s) ds = 1. Replace $, in (2.38) by

,m

(2.45)

Since L, is an algebra, B,,(r)f= .)f defines a bounded linear operator on L,, and the differentiability of p ensures the existence of qk (see Problem 23 of Chapter 1). Letting n+ 00 in (2.43) gives

!w(x* I t ) = - c flP((u* - t - srr)n)qdx,). 1-1

(2.46) .[ n J d U d ) exp { - T H k fr(&(s)) ds - L e i I

we note that (2.46) implies (2.18) and that

(2.48)

Part (b) now follows by Lemma 2.6. 0

The following proposition is useful in reducing the study of (2.1) with unbounded fi, to the bounded case.

3. CONVERCENCL 321

2.10 Proposition Let a be measurable, and suppose inf, a(z) > 0. Let Z be an €-valued stochastic process, let q satisfy

(2.49) r ' a ( Z ( s ) ) ds = t ,

lim,4m ~ ( r ) = a as., and define

(2.50) Z"(0 = Z(V(N.

Then Z is a nonanticipating solution of (2.1) if and only if Z" is a nonanticipating solution of

(2.51)

Proof. If Z satisfies (2.1). a simple change of variable verifies that Z" satisfies (2.51). Assume 2 is a nonanticipating solution, and let (9,) be the family of a-algebras in Lemma 2.6. Since the z(l) form an increasing family of {Y,)-stopping times, and for each s, q(s) is a {Y,,,,}-stopping time, Proposition 8.6 of Chapter 2 gives that z'(s) = r(rl(s)) is a (9,)-stopping lime. Consequently, by Lemma 2.6, 2' is a nonanticipating solution of (2.5 I) .

0 The converse is proved similarly.

3. CONVERGENCE

We now consider criteria for convergence of a sequence of processes Zfn' satisfying

(3.1)

where Yf') is a process with sample paths in D,,[O, 00). We continue to assume that the (&., r k ) are complete and separable. Relative compactness for sequences of this form is frequently quite simple.

3,l Proposition Let 2"' satisfy (3.1). If { Yf') is relatively compact in D,,[O, 00) and fi; = sup, supz Pr ' (z) < 03, then {Z?'} is relatively compact in DE,[O, a), and hence if {I".'} is relatively compact in flk D,[O, 00) and sup,, sup, pl"'(z) < 00 for each k, then {Z'"'} is relatively compact in nk D E b L o v O0).

Proof. The proposition follows immediately from the fact that

(3.2) wt(zp~. 6, r) s w'( vp. /r, s, rS, T )

322 RANDOMTIMECHANGES

and

(Recall that we are assuming the (Ek, rk) are complete and separable.) 0

We would prefer, of course, to have relative compactness in DEIO, 00) where E = n,, Ek, but relative compactness of {Yea)} in DEIO, 00) and the boundedness of the B1;") do not necessarily imply the relative compactness of {Z(")} in &[O, 00). We do note the following.

3.2 Proposition Let {Z(")} be a sequence of processes with sample paths in D,[O, ao), E = f l i t Ek. If Z(") =+ 2 in flk D,,[O, 00) and if no two components of 2 have simultaneous jumps (i.e., if P{Zk(r) # Z&-) and Z,(r) # Z,(t-) for some t 2 0) = 0 for all k # I), then 2'"' =e 2 in Ds[O, 00).

Proof. The result follows from Proposition 6.5 of Chapter 3. Details are leR 0 to the reader (Problem 5).

We next give the analogue of Theorem 1.5.

3.3 Theorem Suppose that for k 3: 1, 2, ..., U,, defined on (Q S, f), has sample paths in D,[O, a), is nonnegative, bounded, and continuous on E = nl E l , and either U, is continuous or flk(z) > 0 for all z E E. Suppose that for almost every o E Q,

(3.4)

has a unique solution. Let { Y'"'} satisfy Y(*)=* Y in fl, D,[O, a), and for k = 1, 2, ..., let /$) be nonnegative Bore1 measurable functions satisfying sup, sup,. pp)(z) < Q) and

(3.5)

for each compact K c E. Suppose that Z(") satisfies

and that Wfr) satisfies

(3.7)

PmoC The proof is essentially the same as for Theorem 1.3, so we only give a sketch. We may assume limn-m Y("' = Y 8.s. The estimates in (3.2) and (3.3) imply that if { ~ ' " ' ( w ) } is convergent in n k D,[O, 00) for some w E a, then { P ' ( w ) } and { W"(w)) are relatively compact in k D,[O, 00). The continuity and positivity of Ok imply that any limit point P (0) of must satisfy

(3.8)

(If V, is not continuous, then the positivity of p k implies = Yk-(o, yo &(2(0 , s)) ds) for almost every r 2 0. See Problem 6.) Since the solution of (3.4) is almost surely unique, it follows that limndm 2'") = 2 and limn4m W(") = 2 in nk D,[O, a)) 8.s. 0

The proof of Theorem 3.3 is typical of proofs of weak convergence: compactness is verified, it is shown that any possible limit must possess certain properties, and finally it is shown (or in this case assumed) that those properties uniquely determine the possible limit. The uniqueness used above was strong uniqueness. Unfortunately, there are many situations in which weak uniqueness for nonanticipating solutions is known but not strong uniqueness. Consequently we turn now to convergence criteria in which the limiting process is characterized as the unique weak, nonanticipating solution.

We want to cover not only sequences of the form (3.6) and (3.7) but also solutions of equations of the form

(3.9)

where r'") is a rapidly fluctuating process that "averages" /?:"'in the sense that

(3.10)

The following theorem provides conditions for convergence that apply to all three of these situations.

3.4 Theorem Let Y("), n = 1, 2, ..., have values in nk D,[O, 00). let (9:') be a filtration indexed by [O, 00)" satisfying 9:' 3 o( Yt'(sk): sk s u,, k = 1, 2, ...), and let P b ( t ) , t 2 0, be a nondecreasing (componentwise) family of {rJ?'}-stopping times that is right continuous in 1. Define

(3.1 I ) Z:ll'(t, = Y:I)(rl"'cr)).

Suppose for k = I, 2, . . . that { G(c)} is a strongly continuous semigroup on L, c is(Ek) corresponding to a Markov process V,, and L, is convergence determining, that &: E + [O, a)) is continuous, and that either /Ik > 0 or V, is continuous.

(3.13)

for each k = 1, 2, . . . and t 2 0.

(a) If (Y("), 2'"') ( Y , 2) in n k D,[O, 00) x n k D4[0, a), then Z is a

(b) Suppose that for each E, T > 0 and k = 1, 2, .. . there exists a nonanticipating solution of (2.1).

compact K: T c f& such that

(3.14) inf P(Z:"'(t) E K:, for all t I; T } 2 1 - E.

If Y(")* Y in n k D,[o, a), and (2.1) has a weakly unique nonanticipating solution 2, then P"=+ Z in n k D,[o, a).

0

3.5 Remark (a) Note that (3.12) implies that the finite-dimensional distributions of Y'") converge and that the & are conditionally independent given Y(0). See Remark 8.3(a) of Chapter 4 for conditions implying (3.12).

(b) If the Y:") are Markov processes satisfying

(3.17)

It follows that z k ( l ) = q ( ? k ( t ) ) Or Y;(fk(t)). We need z k ( t ) 3 &(fk(t)). I f & is continuous, then (2.1) is satisfied; or if > 0, then the fact that f k is

3. CONVERGENCE 325

(strictly) increasing and &(t) and Yk(rk(f)) are right continuous implies (2. I ) is satisfied.

To see that 2 is nonanticipating, note that with the parameters as in (2.19)

Observe that the rI are continuous and that P( Yk(t) = V,(t --)} = 1 (cf. Theorem 3.12 of Chapter 4) for all t . Consequently all the finite-dimensional distributions of (Y'"), TI")) converge to those of (Y, T). By Lemma 2.7, Z is a nonanticipating solution of (2.1).

(b) By part (a), i t is enough to show that { (Y'") , Z'"))} is relatively compact, since any convergent subsequence must converge to the unique nonanticipating solution of (2.1). By Proposition 2.4 of Chapter 3, i t is enough to verify the relative compactness of {Z:"'). Let

326 RANDOM TIM€ CHANCES

The monotonicity of yp) and rp) and (3.14) imply the convergence in (3.13) is uniform in t on bounded intervals. For 6, T > 0, let

(3.20) qr)(6, T) = sup ($)(t + 6) - rt ) ( t ) ) + sup (rl'"(t) - T ~ ) ( c - ) ) .

Note that by the uniformity in t in (3.13) and (3.14), as n+ 00 and 6-0, qp)(6, T) 4 0. Finally

(3.21) w'(Zjll), 6, T ) 5; w'(Yt), #'(& T), tr'(T))

(see Problem 7), and hence for E > 0 the relative compactness of { Up)} implies

(3.22)

tsr I S T

lim lim P { W ' ( Z ~ ) , 6, T ) > E } d - 0 m-+oo

4 Jim lim P{w'(Yp', q t y d , TI, T ~ ) ( T ) ) > E ) &-.a * - O D

-0

and the relative compactness of (2:")) follows. 0

3.6 Corollary Let Y, , Y, , . . . be independent Markov processes (as above), let Pk: E + [O, a) be continuous and bounded, and assume either /.Ik > 0 or & is continuous. Then (2.1) has a weak, nonanticipating solution.

Proof. Let Y'*) = Y and W(") satisfy (3.7) with h, = I/n. Then { Wp') is relatively compact by essentially the same estimates as in the proof of Proposition

0 3.1. Any limit point of {Wen)} is a nonanticipating solution of(2.1).

4. MARKOV PROCESSES IN 2'

Let E be the one-point compactification of the &dimensional integer lattice Z', that is, E = Z' u {A). Let p,: itd+ [O, a), I E Z', P,(&) c oo for each & E Z', and forjvanishing off a finite subset of 2'. set

C hlxKJ(x + 0 -f(x)), x E Z',

io, x = A. Af(x) = J (4.1)

Let x , I E Z', be independent Poisson processes, let X(0) be nonrandom, and suppose X satisfies

4. MARKOV PROCESSES IN 2' 327

and

(4.3) X ( t ) = A, t 2 T ~ ,

where r m = inf { t : X ( t - ) = A}.

4.1 Theorem (a) Given X(O), the solution of (4.2) and (4.3) is unique. (b) X is a solution of the local-martingale problem for A. (Cf. Chapter

4, Section 7. Note, we have not assumed Afis bounded for eachJE 9 ( A ) . If this is true, then X is a solution of the martingale problem for A.)

(c) If 2 is a solution of the local-martingale problem for A with sample paths in Ds[O, 00) satisfying R(t) = A for I 2 rm ( T ~ as above), then there is a version X of 2 satisfying (4.2) and (4.3).

Proof. (3 Let X0(t) I X(0) and set

(4.4)

Then if rk is the kth jump time of xk , Xh(t) = X, - , ( t ) for r < T ~ . Therefore

(4.5)

exists and X satisfies (4.2). We leave the proof of uniqueness and the fact that limh-,m rk = T~ to the reader.

(b) Let a(x) = I + El /?,(x)and

X ( t ) = lim X& t -= lim T k , k-m k - m

(cf. Proposition 2.10). Then Xo(t) 3 X(q(t)) is a solution of

(4.7) Xo(t) = X ( 0 ) + 1 IV, I

where /I? s Pl/a. Note Z, < 1. If X o is a solution of the martingale problem for A' (defined as in (4.1) using the P;), then by inverting the time change, we have that X is a solution of the local-martingale problem for A. For z E (Z +)", let

and set

(4.9)

328 RANDOM T I M CHANCES

Sirice Z is the unique solution of (4.9), it is nonanticipating by Theorem 2.2. Consequently, by Theorem 2.8(a), 2 is a solution of the martingale problem for

B = {( n I;, c fl:cfL(' + eL) -h) n A): I c z', 1 finite,I; E w + ) . l e l L Iek I

The bpclosure of B contains (A G,, /l;(f(. + el) -f)) where f is any bounded function depending only on the coordinates with indices in I (I finite). Consequently,

(4.10) f X(0) + C E X r ) ( 1 6 1

is a martingale for any finite I and any f B 9 ( A 0 ) . Letting I increase to all of Zd, we see that Xo is a solution of the martingale problem for A'.

(c) As before let

(4.1 1)

Then xo(t) E z(ij(t)) is a solution of the martingale problem for A'. But A' is bounded so the solution is unique for each f (0 ) . Consequently, if X(0) = R(O), then by part (b), Xo must be a version of Ro and X must be a version of 8. 0

5. DIFFUSION PROCESSES

the one-point compactification of Fad. For k = I , 2, . . . , measurable, a, E W', and suppose that for each compact IaLl2Pk(x) < OD. Thinking of the or, as column vectors,

Let F: Rd-+ R' be measurable and bounded on compact sets. ForfE C,"(Rd)), extendfto E by settingJ(A) = 0, and define

5. DIFFUSION PROCESSES 329

Let w, i = 1, 2, . . . , be independent standard Brownian motions, let X ( 0 ) be nonrandom, and suppose X satisfies

and

(5.4) X(r ) = A, t 2 T,,

where r m = inf { t : X ( f - ) = A]. The solution of (5.3) and (5.4) is not in general unique, so we again employ the notion of a nonanticipating solution. In this context X is nonanticipating if for each t 2 0, Wl = %(zit) + .) - q . ( r i ( f ) ) , i = I , 2, . . . , are independent standard Brownian motions that are independent of F,X.

5.1 solution of the martingale problem for A.

Theorem If X is a nonanticipating solution of (5.3) and (5.4), then X is a

5.2 Remark (a) Note that uniqueness for the martingale problem for A

(b) A converse for Theorem 5.1 can bc obtained from Theorem 5.3 0

implies uniqueness of nonanticipating solutions of (5.3) and (5.4).

below and Theorem 3.3 of Chapter 5.

Proof. The proof is essentially the same as for Theorem 4.I(b). 0

To simplify the statement of the next result, we assume r, = n~ in (5.3) and (5.4).

5.3 Theorem (a) If X is a nonanticipating solution of (5.3) for all f < co (ie., rm = a)), then there is a version of X satisfying the stochastic integral equation

(5.5) y(r) = Y(o) + f aI [,/mi d ~ L s ) + l F ( Y ( s ) ) ds. i = I 0 .

(b) If Y is a solution of (5.5) for all t < a), then there is a version of Y that is a nonanticipating solution of (5.3).

Proof. (a) Since X is a solution of the martingale problem for A , (a) follows from Theorem 3.3 of Chapter 5.

Let q, i = 1, 2, . . . , be independent standard Brownian motions, independent of the Bi and Y . (It may be necessary to enlarge the sample space to obtain the q. See the proof of Theorern 3.3 in Chaptcr 5.)

(b)

330 RANDOM TIME CHANCES

Let

and let

(5.7)

Define

ydu) = inf {t : rj,(V(s)) ds > u}, u s Ti(m). 0

Since yi(u) is a stopping time, is a martingale by the optional sampling theorem, as is W:(u) - u. Consequently, W, is a standard Brownian motion (Theorem 2.1 1 of Chapter 5). The independence of the W, and the stopping propertiesof the rl follow by much the same argument as in the proohf the independence of the & in Theorem 2.8(b). Finally, since fo d , Y ( s ) dBAs) is constant on any interval on which pi( Y(s)) is zero, it follows that Y is a solution of (5.3). 0

The representations in Section 4 and in the present section combine to give a natural approach to diffusion approximations.

5.4 Theorem. Let /3!’”: Rd+ [O, a), a1 E IRd, i = 1,2, ..., satisfy

(5.9)

for each compact K c Rd, and let A, > 0 satisfy limn-a A, = 00. Let q , i = 1, 2, , . . , be independent unit Poisson processes and suppose A’, satisfies

(5.10)

Define Wy)(u) = I ; ‘ I2 ( x(In u) - I, u) and

(5.1 1) F,(x) = A,!” 1 Q,@’(x).

Let pi: Rd-+ [O, a), i = I , 2, . . . , let F: Rd-+ R’ be continuous, and suppose for each compact K c Iw’ that

(5.12)

I

lim sup I/?f‘)(x) - fi,(x)l = 0, i = 1, 2, ..., n+m r s K

(5.13) lim sup I F,(x) - F(x) I = 0, n4 .n x r K

5. DIFFUSION PROCESSEJ 331

and

(5.14)

Suppose that (5.3) and (5.4) have a unique nonanticipating solution and that X,,(O)-+ X(0) . Let T: = inf {r : I X,(t) I 2 a or I X,(r -) I 2 u } and r, = inf { r : I X( t ) I 2 a}. Then for all but countably many a 2 0,

(5.15) X,(. A T:) * X( * A ra).

If Iim,-.,,, T, = 00, then X, - X .

5.5 Remark More-general results of this type can be obtained as corollaries to Theorem 3.4. 0

Proof. Note that

(5.16) X,(r) = X,(O) + C ai WP' I

It follows from (5.12). (5.13), (5.14). the relative compactness of { Wf"}, and (5.16), that {X,( * A r:)) is relatively compact (cf. Proposition 3.1). Furthermore, if for a. > 0 and some subsequence {q}, X,,( * A r:;) =. KO, then setting q, = inf { r : I Y&) I 2 a or I X,(t -) I 2 a}, (XJ A r3, r:l) 9 ( Y,,,(. A q,), q,) in DRd[O, 00) x [0, 003 for all a < a. such that

(5. I 7)

Note that the monotonicity of q, implies (5.17) holds for all but countably many a.

Since uo is arbitrary, we can select the subsequence so that {(X,,(. A T:'), 1:'))

converges in distribution for all but countably many a, and the limit has the form ( Y ( . A a,), q,) for a fixed process Y with sample paths in &LO, 00) (q, as before). (We may assume that Y(t) = A implies Y(s ) = A for all s > 1.) By the continuous mapping theorem (Corollary 1.9 of Chapter 3). Y satisfies

(5.18) Y(t A q,) = Y(0) + ai bf( (r A '-Pi( Y(s)) ds) + LA "F( Y(s)) ds.

Here (5.14) allows the interchange of summation and limits. It follows as in the proof of Theorem 3.4 that Y is a nonanticipating solution of (5.3) and (5.4) and hence Y has the same distribution as X. The uniqueness of the possible limit point gives (5.15) for all a such that qs = q, as. The final statement of the theorem is left to the reader. 0

0

332 RANDOM T I M CHANGES

Equations of the form of (5.10) ordinarily arise after renormalization of space and time. For example, suppose

(5.19)

and set X,,(t) = n- '''U,,(nt). Then X , satisfies

6. PROBLEMS

1. Let W be standard Brownian motion. (a) Show that for 0 c a < 1,

and for a 2 1,

' 1 (6.2) ds = 00 a s , I > 0.

(b) Show that for a 2 1 the solution of

is unique, but it is not unique if 0 < a < 1. (c) Let 0 < a < 1 and yo = sup {t < 100: W(t) = O}. Let r(r) satisfy

Show that X( t ) = W(z(t)) satisfies (6.3), but that it is not a solution of the martingale problem for A p. {(J ) I x l y " ) : J ~ Ccp)(W)}.

2. Let Y, and Ys be independent standard Brownian motions. Let PI and P2 be nonnegative, measurable functions on R' satisfying &(x, y) S

6. PROOLEMS 333

K(I + xz + y’). Show that the random time change problem

(6.5)

is equivalent to the martingale problem for A given by

(6.6) A = { ( A P I / A + bl f y y ) : S ~ C,‘(W’),),

that is, any nonanticipating solution of (6.5) is a solution of the martingale problem for A, and any solution of the martingale problem for A has a version that is a weak, nonanticipating solution of (6.5).

3. State and prove a result analogous to that in Problem 2 in which Y, and Y, are Poisson processes.

4. Let Y be Brownian motion,

(6.7)

and

(6.8)

Show that

(6.9)

has no solution but that

(6.10)

does. In the second case, what is the (strong) generator corresponding to Z?

5. Prove Proposition 3.2.

6. For Y,(r) = [ t ] and YZ(t) = t , let (Zy’, Z?’) satisfy

Z:“)(C) = Y, (I; I - n - ‘)J( I - Z?)(s)) v 0 d s ) *

- Z?l(s)) v 0) ds) ,

0 (6.1 1)

334 WDOM TIME CHANGES

and let (Z, , Z,) satisfy

ZAt) = Y, (rAl - Z2(s))V0 ds), 0

(6.12)

Z,(t) = Y2(L(Z,(? + J-) ds).

hat limn-. (Z','", Z?)) # (Z, , Z2).

7. Let T(C) be nonnegative, nondecreasing, and right continuous. Let y E

D,[O, 00) and t = y(r( a)). Define

(6.13) q(6, T ) = SUP (T(C + 6) - T((t)) + SUP (T(f) - T(C-)), rsr tsr

and y( r ) = inf {u: T(U) 2 1 ) . Show that if 0 s t l c t2 and c, - c 1 > q(6, T), then y(t,) -- y(tl) > 6, and that

(6.14)

8. Suppose in (4.2) that 1 III&(x) 5 A + E l x l . Show that r m = 00.

9. Let W and Y be indepcndent, W a standard Brownian motion and Y a unit Poisson process. Show that

WYZ, 4 7-1 5 W I Y , N, T), 7(T)).

(6.15)

and

(6.16) 2"(t) = p- dW(s)

have the same distribution, that {Z,} converges as., but (2,) does not converge a.s.

10. Let E = {(x, y): x, y 2 0 , x + y s I} . For JE C"(&), define .41= x(1 - x)j"= - 2xyl;, + y(1 - y)f,,. Show that if X is a solution of the martingale problem for A, then X satisfies (5.3) with at = 0, i 2 4.

11. Letfand g be locally absolutely continuous on R. (a) Show that if h is bounded and Bore1 measurable, then

(6.17) O ( d d ) g W dz = L, Mu) du, o(b)

0, b E R,

with the usual convention that j: f ( z ) dz = -fi f ( z ) dz if b < a. Hint: Check (6.9) first for continuous h by showing both sides are locally absolutely continuous as functions of b and differentiating. Then apply a monotone class argument (see Appendix 4).

(b) Show that if A E

(6.1 8)

7. NOTES 335

a(R) has Lebesgue measure zero, then

In particular, for each a, m({g’(z) # 0 ) n { g ( z ) = a ) ) = 0. Show that if g is nondecreasing, then f 0 g is locally absolutely continuous.

(c)

(d) Define

(Note that m(W - ( z : f ‘ ( g ( z ) ) and g’(z) exist} u ( 2 : g’(r) = 0 ) ) = 0.) Show that J 0 g is locally absolutely continuous if and only if h is locally L’, and that under those conditions

(6.20)

Letf(r) = f i and g(t) = t Z cos2 ( l / f ) . Show that/and g are locally absolutely continuous, bu t fo g is not. Hint:

d - f(g(z)) = h(r) a.e. dz

(e)

Show tha t f a g does not have bounded variation.

12. Let /I be a nonnegative Borel measurable function on [O, m) that is locally L!. Define y ( t ) = inf (u: 6 /I(s) ds > 1 ) .

(a) (b) Show that

(6.21)

for all 0 I a < b. Show that if g is Borel measurable and /Ig is locally L’, then

(6.22)

Show that y is right continuous.

p(4 ds = rx,asy,,,<*, d

(c)

7. NOTES

Volkonski (1958) introduced the one-parameter random time change for Markov processes. See also Lamperti (1967b). Helland (1978) gave results similar to Theorem 1.5 with applications to branching processes (see Chapter 9, Section I).

336 RANDOM TIM€ CHANCES

The multiparameter time changes were introduced by Helms (1974) and developed in Kurtz (1980a). Holley and Stroock (1976) use a slightly different approach.

Applications of multiparameter time changes to convergence theorems are given in Kurtz (1978a. 1981~. 1982). See Chapters 9 and 11.

Any diffusion with a uniformly elliptic generator with bounded coeficients can be obtained as a nonanticipating solution of an equation of the form of (5.3) with only finitely many nonzero al. See Kurtz (198Oa).

7

Let (, , (, , . , . be independent, identically distributed random variables with mean zero and variance one. Define

(0.1)

A simple application of Theorem 2.6 of Chapter 4 and Theorem 6.5 of Chapter 1 gives Donsker's (1951) invariance principle, Theorem 1.2(c) of Chapter 5, that is, that X,, W where W is standard Brownian motion. One noteworthy property of X , is that it is a martingale. In Section I we show that the invariance principle can be extended to very general sequences of martingales.

Another direction in which the invariance principle has been extended is to processes satisfying mixing conditions, that is, some form of asymptotic independence. A large number of such conditions have been introduced. We consider some of these in Section 2 and give examples of related invariance principles in Section 3.

Section 4 is devoted to an extension of the results of Section I allowing the limiting process to be an arbitrary diffusion process.

Section 5 contains recent refinements of the invariance principle due to Komlos, Major, and Tusnidy (1975, 1976) who showed how to construct X n and W on the same sample space in such a way that

lntl

X,(r) = H - "2 c &. k = I

INVARIANCE PRINCIPLES A N D DIFFUSION APPROXIMATIONS

337



338 INVARIANCE PUINCIPUS AND DIFFUSION APPROXlMATlONS

1. THE MARTINGALE CENTRAL LIMIT THEOREM

In this section we give the extension of Donsker's invariance principle to sequences of martingales in &Id. The convergence results are based on the following martingale characterization of processes with independent increments.

1.1 Theorem Let C = ((c,,)) be a continuous, symmetric, d x d matrix- valued function, defined on [O, a), satisfying C(0) = 0 and

Then there exists a unique (in distribution) process X with sample paths in C,,[O, co) such that Xi, i = 1, 2, . .., d, and X,X, - c,,, i, j = 1, 2, . . ., d, are (local) martingales with respect to {P:}. The process X has independent Gaussian increments.

Proof. As in the proof of Theorem 2.12 of Chapter 5, if X is such a process, then for 0 E Rd

(1.2) f(t, X) = exp { i0 X ( t ) + +e c(t)e}

is a martingale, and hence

which implies X has independent Gaussian increments and determines the finite-dimensional distributions of X .

To obtain such an X set

Note that (1.1) implies

(take {, =I I , {, - * 1, and variation and ctf can be written as

= 0 otherwise), and hence is of bounded

1. THE MARTINGALE CENTRAL L l M n THEOREM 339

where D(s) = ((d,ks))) is nonnegative definite. Let D'"(s) denote the symmetric nonnegative-definite square root of D(s), let W be d-dimensional standard Brownian motion, and set

(1.7) M ( t ) = W o w .

Then

(1.8) X( t ) = D dM l is the desired process. 0

1.2 Theorem process and that, for each 8 E W d and /E C?(R),

Let C be as in Theorem 1.1. Suppose that X is a measurable

is an { 9:}-martingale, where

(1.10)

(1.1 1) E[X(r)X(r)'] = ~ ( r ) .

cr(t) = e - c(t)s. Then X has independent Gaussian increments with mean zero and

1.3 Remark Note that it is crucial that (1.9) be a martingale with respect to (9:) and not just with respect to (9:") = a(@ . X(s): s 5 r } . See Problem 2.

0

Proof. The collection off for which (1.9) is an {Sf}-martingale i s closed under bpconvergence. Consequently,

(1.12)

is an (Sf}-martingale, and hence, by Itii's formula, Theorem 2.9 of Chapter 5,

(1.13) exp (ie . X ( t ) + )c&)}

is an (.FP:)-martingale. The theorem follows as in the proof of Theorem 1. I. 0

1.4 Theorem For n = 1, 2, ..., let {9:} be a filtration and let M, be an {SF:)-local martingale with sample paths in DRIIO, 00) and M,(O) = 0. Let A, = ((A:')) be symmetric d x d matrix-valued processes such that A:' has sample paths in DRIO, GO) and A&) - A&) is nonnegative definite for I > s 2 0. Assume one of the following conditions holds:

340 INVARIANCE PRINCIPUS AND DIFFUSION AMOXWllOM

(3

(1.14)

and

For each T > 0,

lim E sup M,(t) - M,(t -1 I = 0 n-m [ i s T I

n-m [ i s T 1 n-m c I ~ T 1

(1.15) A! = [MI,, Mi].

(b) ForeachT>Oandi , j= 1,2 ,..., d,

lim E sup J Ai’(f) - A:’(‘(t -) J = 0,

lim E sup I M,(t) - M,(t -)I2 = 0,

(1.16)

(1.17)

andfori , j= 1,2 ,..., d, (1.18) M;(t)M!(f) - A,’(t)

is an {.F:}-local martingale.

Suppose that C satisfies the conditions of Theorem 1.1 and that, for each t r O a n d i , j = 1,2 ,..., d,

(1.19) A!i’(t) + C&)

in probability. Then M, 3 X, where X is the process with independent Gauss- ian increments given by Theorem 1.1.

1.5 Remark In the discrete-time case, let {tz: k = 1, 2, ...} be a collection of Rd-valued random variables and define

(1.20)

for some a,,+ 00. In condition (a)

(1.21) 4- 1

(considering the (: as column vectors), and for condition (b) one can take

(1.22)

where 9; 5 a((;: 1 5 k), Of course M, is a martingale if EL€; 19; - I J = 0. 0

Proof. Without loss of generality, we may assume the Mn are martingales. I f not, there exist stopping times T, with P ( r , < n} 5 , - I such that M , ( . A r , ) is a martingale and A,(.Ar,) satisfies the conditions of the theorem with M,

1. T M MARTINGALE CENTRAL LIMIT THEOREM 341

replaced by Mn( * A r,). Similarly, under condition (b) we may assume the processes in (!. 18) are martingales.

(a] Assume condition (a). Let

(1.23) q, = inf { r : Af ( t ) > c,,(t) + I for some i E { I , 2, ..., d}}.

Since ( I .19) implies q, A 00 in probability, the convergence of M, is equivalent to the convergence of &, = M,(. A q,).

Fix 8 E Rd and define

( I .24)

(1.25)

and

(1.26)

Let

(1.28) y = max ( k : t k < q,A(t + s)]

and

312 INVAIIANCE PRINCIPLES AND DIFFUSION AWROXM4llOM

Setting AY,(u) = Y,(u) - Y,(u-), and letting max (rk+, - tk ) - 0,

Note that the second term on the right is bounded by

(by the definition of q"), and hence

where C, depends only on 11/'11, II/"II , and IIf"'[l. In particular (1.33) can be extended to all J(inc1uding unbounded j ) whose first three derivatives are bounded.

Let cp be convex with Cp(0) = 0, limx-,w q ( x ) = 00, and cp', cp". and 9'" bounded. Then

1. THE MARTINGALE CENTRAL LIMIT THEOREM 343

where

+ sup (A:((r + a)Aq,,--) - A:(f Aqm-)) . 1'5 r 1

(1.37) lim lim E[yn(a)] = lim(C,2 + 211fllCf) sup (ce(f + 6) - c&)) d+O q-.m d - 0 I s +

= 0,

so for each /E C?(W"), {f(Y6)} is relatively compact by Theorem 8.6 of Chapter 3. Consequently, since we have (1.34), the relative compactness of { U,} = (0 . a,} follows by Theorem 9.1 of Chapter 3. Since 0 is arbitrary, {a,} must be relatively compact (cf. Problem 22 of Chapter 3).

The continuity of ckr) and (1.19) imply

(1.38) ?$''(V(u-)) dA!(u) - tf"(v(u)) dc,(u)-+ 0 (1 . (I +b) A L)

in probability uniformly for y in compact subsets of DRIO, 00). Consequent- ly the relative compactness of { U.} implies

344 INVAIIIANCE r n w x u s AND DIFFWON A~OXIAMTIOM

The first two terms on the right of (1.32) go to zero in L' and it follows easily (cf. the proof of Theorem 8.10 in Chapter 4 and Problem 7 in this chapter) that if X is a limit point of {G,,}, then (1.9) is an {.Ff}-martingale. Since this uniquely characterizes A', the convergence of {A?,,) follows.

(b) The proof is similar to that of part (a). With q,, and fi,, defined as in (1.23), we have

( 1.40) Aif(t A q,) S c,Xt) + 1 + SUP ( A ~ ( s ) - A:(S -)), S S I

and the third term on the right goes to zero in L' by (1.16). Setting @(t) = #(f Aq,,), note that

and to apply Theorem 8.6 of Chapter 3, fix T > 0 and define

( 1.42)

Since

(1.43) yn(6) s

1

~"(6) = sup 1 (AAt'((r + d) - JAW). c L T l = l

and since (1.16) implies the right side of (1.43) is convergent in f?, we conclude that

(1.44) 1

Iim lim &[y,,(d)] = lim sup C (clt'(r + 6) - c , ~ t ) ) = 0. d - 0 n+co 4 - 0 1st 1 = I

Let X be any limit point of {a,,}. By (1.17), X is continuous. Since for each T > 0, sup,, E[ I fi, ,(T) 1'3 < 00, {f i , , (T)} is uniformly integrable, and hence X must bc a martingale (see Problem 7). Since XI X, - ci, is the limit in distribution of a subsequence { fia a{, - {A}, we can conclude that it is also a martingale if we show that {&!JZ")M&(T) - A%(")} is uniformly integrable for each T. Since (1.40) and (1.16) imply that {J$(T)) is uniformly integrable (recall I AiJ(T)I 5 #A"(T) + Ai'(7'))). it is enough to consider {ab,(T)&&(T)}, and since I i@h(T)&&(T)I s #@$QJ + fi&(T)2), it is enough to consider { f ik (T)2} . Since fiik(T)z =+ XXT) , {fi:(T)'} is uniformly integrable if (and only if) E[&!,,(T)2J -+ E[X, (T)2J , that is, if

2. MEASURES OF MIXING 345

and

( I .47)

Since

fii(T A TE)2 5 2 ( a + sup I f i i ( s ) - f i i ( s - ) 12 , ( I .48)

( f i L ( T A T:)} is uniformly integrable by (1.17). For all but countably many a and T, (r:, , @,,(TAT:)) * (re, X‘(T A P)) and, excluding the countably many a and T,

(1.49) &[XXT A t‘)’] = lim €[fi:( T A r:,)’]

T‘ = inf { 1 : X,(r)’ > a) .

) IS T

k-a,

= lim E[Ai;(T A rzJJ k - m

= E[c,XT A P)].

Letting a-+ 00 we have (1.45). and it follows that X , X , - c,, are martingales for i , j = 1, . . . , d, and that X is the unique process characterized in Theorem 1.1. Therefore M, * X. 0

2. MEASURES OF MIXING

Measures of mixing are measures of the degree of independence of two 0-

algebras. Let (n, F, P ) be a probability space, and let 48 and X be sub-a- algebras of 9. Two kinds of measures of mixing are commonly used. The measure of unqorm mixing is given by

(p(f4I.w) = sup sup I P(A

= sup IIP(A I *#) -

A s 9 Bci?” (2.1)

P ( B ) > O

,469

where 1) * 11, denotes the norm for P(Q. 9, P). The proof of equality of the two expressions is left as a problem. The measure of strong mixing is given by

a($, .W) = sup sup I P ( A B ) - P(A)P(B)J (2.2) A s * B c l p

= f SUP EC I P(A 1-m - P(A)ll A s s

= f SUP E[ I P(RI 9) - P(B) I] B C x

= + SUP lIP(A I .*I - P(A)II I *

A . 9

346 INVARIANCE PRINCIPLES AND DIFFUSION APPROXIMATIONS

Again the equality of the four expressions is left as a problem.

general definition. For 1 5 p $ 00 set A comparison of the right sides of (2.1) and (2.2) suggests the following

4qg I J1") = SUP I1 P(A I - P(A)lI, ' A s 9

(2.3)

Note that cp = cpm and a = +cp,. Let E , and E2 be separable metric spaces. Let X be El-valued and g-

measurable, and let Y be El-valued and #-measurable. The primary application of measures of mixing is to estimate differences such as

(2.4)

where pr and f lu are the distributions of X and Y. Of course if X and Y are independent then (2.4) is zero. We need the following lemma.

2.1 Lemma Let pl and pz be measures on 9 , and let lipl - p21j denote the total variation of p1 - p 2 . Let r, s E [l, 003, r W 1 + s-l = 1. Then for g in w, 3, P I + c12h

Proof. Then

Let S, be the Radon-Nikodym derivative of P, with respect to pl + p2.

Ilp1 - P2 I1 3 SUP (CIM - P Z ( 4 + SUP (CId4 - P M ) A6s A i l

(2.6)

and

2.3 Remark Note that for q > 2 we may select s = q/p so that (2.8) becomes

(2.9) I ECZY1 - ECZlEC vl I s 4 v y w I Jf)llZll,Il YII, - 0

(2.1 I ) I E[ZY] - E[Z]E[ Y] I = 2 djI1 - is I Z d 4 5 2l"cp;"(Y 1 #)[I Yll;"(E[ I z I"] + E[ I2 " ] E [ Y])"S

s 2cP:'vI ~ ~ l l ~ l l s P l l Yll,.

since both E[ I Z YY] and E[ 12 1qE[ Y] are bounded by E[ I Z rp]l'PII Y 11,. For general Y, apply (2.11) to Y + and Y - and add to obtain (2.8). Note

that It Y + 11, + It Y - 11, s 211 I'll, for all q, and for q < 2 this can be improved to l lY+lI*+ I I ~ - I l , ~ 2 ~ - L l l y I I , . 0

2.4 Corollary UQ, 9, P) ,

(2.12)

Let 1 5 r , s 5 00. r - ' + s-' = 1. Then for real-valued 2 in

ECIE[ZlJf'l - ECZ111 5 8~fWIJf')llZIl,.

Proof. Let Y, be the indicator of the event ( E [ Z ( X ] - E[Z] 2 0) and Y, = 1 - Y, . Then

(2.13) E C I E C Z I . ~ I - ECZIII = E [ E [ Z ( M J Y , - €'[ZJYlJ - E[E[Z)X]Yz - E[ZJYJ

= E [ Z Y , J - E [ Z ] E [ Y ~ ] + I E [ Z Y , ] - E[Z]E[Yz]I

and (2.12) follows from (2.8). 0

2.5 Corollary Let 1 5 u, u, w S 00, u - ' + u - ' + w - valued Y, 2 with Y in LW(Q X , P) and Z in C(Q, '3, P),

(2.14) IECZY] - E[Z]E[Y]J s 2 " n w A Z + L a""(9. *)ltZl~Il YII,.

= I . Then for real-

348 INVARIANCE FRINCIPUS AND W M O N AmOWMANOM

Proof. By the symmetry of a in Y and JI", it is enough to consider the case . w I; u. Let q = w, p - q/(q - I), s = v/p, and r = s/(s - 1). Note that since u - I + w- ' = u- ' + 4-' 5 1 we must have u 2 p and hence s 2 1, and that u = pr.

BY (2.8)s

(2.15) I ECZY) - ECZlECYl I 5 2'" Z~i''(Y I JQllzll~ll Yll,.

Finally note that

is a decreasing function of p. Replacing p by 1 in (2.15) and q, by 2a gives (2.14). 0

In the uniform mixing case (p = m) much stronger results are possible. Note that for each A E 9

where cp,(Yl~V) is, of course, a constant. We relax this requirement by assuming the existence of an #-measurable random variable Qr such that

(2.18) I P(A I S') - 4.4) I s Q, a.s.

for each A E Y. (See Problem 9.) To see why this generalization is potentially useful, consider a Markov process X with values in a complete, separable metric space, transition function P(t, x, r), and initial distribution Y. Let Y = 9"' = a(X(u): u L t + s) and M = 9, = a(X(u): u 5 t). By the Markov property, for A E 9". there is a function hA such that E [ ~ , , 1 4 t ~ + J = hA(X,+,). Therefore for A E S'",

where

(2.20) P(t + s. x, r)v(dx) r

For examples, see Problem 10.

2. MEASURES OF MIXING 349

2.6 Proposition measurable and satisfies (2.18). Then for real-valued Z in E(R, Y, P),

Let 1 < s s 00 and r - ' + s - ' = 1. Suppose that Q, is Jt"-

(2.21)

(2.22)

and for 1 s p s 00,

I E[Z( &] - E[Z] 1 s 2""'"(E[ I z I'I *] + EC I z y])l/s,

ll~C21*1 - ECzlII. s 2 max (tWrZlts, lW/r~lsltZ~~s),

(2.23)

Proof. A € Y

(2.24)

For

Fix E E J1" with P(E) > 0, and take p, (A) = P(A I E ) and p2(A) = P(A), Then noting that lip1 - pzII s 2P(B)-' JD UJ dP, Lemma 2.1 gives

I I f(E)- I bE[Z I X'] d P - E[Z]

a, 0 > 0, let B = (E[ZlJu'] - ECZ] > 2'Pap. Q, s ar, :I Z I" I Jl"] + E[ I Z Is] 5 p"}. If P(E) > 0, then (2.24) is violated. Consequent- P ( E ) = 0 for all choices of a and PI which implies

(2.25) E[Z I X] - E[Z] 5 2'"UJ"'(E[ I Z 13'3 + E [ I Z r])"", A similar argument gives the estimate for E[Z] - E[Z(M]. Finally (2.21) and

0 the Holder inequality give (2.22) and (2.23).

Proof. Use (2.21) to estimate E[(E[ZIX] - E[Z])Y] and apply the Holder inequality. 0

2.8 Corollary Let 1 < s s a~ and r - ' + s-. ' = I . Suppose that UJ is Jl"- measurable and satisfies (2.t8). Let El and E, be separable. Let X be Y- measurable and E,-valued, let Y be X-measurable and E,-valued, and let px

350 INVARIANCE PRINCIPLES AND DIFFUSION AI*IIOXMTIONS

and py denote the distributions of X and Y. Then for $ in E(E, x E l , dit(E, x E2), px x pu) such that @(X, Y) is in E(f& 9, P),

Proof. Since we can always approximate @ by #,, = n A(# V( -n)) , we may as well assume that @ is bounded, and since the collection of bounded JI satisfying (2.28) is bp-closed, we may as well assume that JI is continuous (see Appendix 4). Finally, if tj is bounded and continuous, we can obtain (2.28) for arbitrary Y by approximating by discrete Y (recall E , is separable), so we may as well assume that Y is discrete.

There exists a A?(€,) x H-measurable function p(y, w) such that E C W , ~ ) l M l = p(y, .) for each Y and E C W , Y)I.Wl = cp(Y(.), -1. (See Appendix 4.) By (2.21)

and hence, since Y is discrete,

Taking expectations in (2.31) and applying the Holder inequality gives (2.29).0

3. CENTRAL UMlT THEOREM FOR STATIONARY SEQUENCES

In this section we apply the martingale central limit theorem of Section 1 to extend the invariance principle to stationary sequences of random variables. Let { &, k E 2) be R-valued and stationary, and define .!Fa = a(Y,,: k S n) and 9” = a(%: k 2 f i ) . Form z 0, let

(3.1) ~ p ( m ) = ( ~ p ( 9 “ + I s a ) *

The stationarity of { %} implies that the right side of (3.1) is independent of n. (See Problem 12.)

3. CENTRAL LIMIT THEOREMS FOR STAllONARY SEQUENCES 351

We are interested in

3.1 Theorem Let ( 5 , k E Z} be stationary with E [ & ] = 0, and for some b > 0 suppose E[ I yk 1' ''1 < 00. Let p = (2 + a)/( 1 + 6) and suppose

(3.3)

Then the series

(3.4)

is convergent and X, 9 X, where X is Brownian motion with mean zero and variance parameter u'.

3.2 Remark (a) The assumption that { Yk} is indexed by all k E Z is a convenience. Any stationary sequence indexed by k = 1, 2, ... has a version that is part of a stationary sequence indexed by Z. Specifically, given {X,, k 2 I } , if (X,} is stationary, then

(3.5) p{Kt, E r l . K + ~ E r2, ..., Y ; + ~ E rm} = ~ ( x , E r , , xz E rZr ..., x, E rm},

(b) By (2.16), for p = (2 + &/(I + b),

I E z, m = I , 2, ..., r,, E .a(&!), determines a consistent family of finite-dimensional distributions.

(3.6)

Consequently the sum on the right of

i s convergent. and M is a martingale. The convergence of the series in (3.4) follows from (2.9). which gives

(3.9) ECYI &I ~ 4 ~ ~ ' " t d ' ( k - 1)IIyiIl,+rlldllz+a.

m m

is a stationary sequence. The sequence

N N

converges in L? (check that it is Cauchy), so

(3.12) E [ ( M ( f ) - M(I - l ) ) ' ] = lim ECG;] N-m

N

= lim ECY3 + 2 1 EC% &+,,,I - ECECK+N+l ts',1'3 N- m ( m u 1

N

- 2 m = 1 E C E [ q + N + I I F l l E ~ ~ + ~ l f ~ l l )

= t9. We have used the stationatity here and the fact that

(3.13) I ECEC8+ N + 1 I S,iEC 8+, t st]] t 5: l E C 8 + N + I ~ ~ 8 + r n ~ ~ l l ] ~

5 4p:c1+s'(N + 1 ) ~ ~ 8 + N + I ~ ~ Z + 4 ~ ~ 8 + m l l 2 + 1 .

The fact that (N + l)cp~"l+s"(N + l ) - + 0 as N - r a0 follows from (3.3) and the monotonicity of rp,,(m). Since { &} is mixing, it is ergodic (see, e.g., Lamperti (1977), page 96), and the ergodic theorem gives

Define M,(t) = n- ' l 'M( [nt ] ) . Then (3.14) gives (1.19).

3. CENTRAL LIMIT THEOREMS F W STATIONARY S€QlJENCES 353

To obtain (1.14). the stationarity of { M ( I ) - M(I - 1) ) implies that for each & > 0,

c 1

s E + I m [ n T 1 P { I M(1) - M(O)l > f i x } dx ,

5 & + T&-'ECIMw - M(0)12X,,M,I)-M(O)I>J.c)l. By Theorem 1.4(a), M, rj X .

Finally note that suplrr I X,(t) - Mn(f)l-+ 0 in probability by the same 0 type of estimate used in (3.19, so X, =?r X .

Now let U),,(m) be a random variable satisfying

(3.16) I P(A 19,) - P(A) I 5 @,(m) a.s.

for each A E P"". Without loss of generality we can assume that for each m, {O,(m)} is stationary and UJ.(m) s I as .

3.3 Theorem and s - ' + r-' = 1. Suppose E[I V,1''*3 < 00,

Let ( Y , , k E E } be stationary with E [ 5 ] = 0. Let I < s 5 00

(3.17)

and

(3.18)

Then the series in (3.4) converges, and X , * X, where X is Brownian motion with mean zero and variance parameter 6'.

3.4 Remark If

(3.19)

then (3.17) and (3.18) hold. 0

Proof. The proof is essentially the same as for Theorem 3.1 using (2.22) to estimate the left side of (3.7) and (2.26) to estimate the left sides of (3.9) and (3.1 3). 0

354 INVARIANCE PRINCIPLES AND DIFFUSION ,wmoxwnoM

4. DIFFUSION APPROXIMATIONS

We now give conditions analogous to those of Theorem 1.4 for the convergence to general diffusion processes.

4.1 Theorem Let a = ((a,))) be a continuous, symmetric, nonnegative definite, d x d matrix-valued function on W’ and let b: R’-+ Rd be continuous. Let

A = {cr, GJE + c ail 8, a , i + c b, a , . t w E ccm(wd)}, and suppose that the C,[O, ao) martingale problem for A is well-posed.

For n = I, 2,. .., let X, and B, be processes with sample paths in DR,[O, a), and let A, = ((A:’)) be a symmetric d x d matrix-valued process such that A:) has sample paths in DJO, 00) and Adt ) - A,(s) is nonnegative definite for t > s 2 0. Set 9: = a(X,,(s), B,(s), Ads): s s c).

Let r: = inf { t : IX,(f)l 2 r or IX,(t-)l 2 r}, and suppose that

14.1 M, EX, - B,

and

(4.2)

are (Sc:}-local martingales, and that for each r > 0, T > 0, and i , j = I,

(4.3)

(4.4)

(4.5)

(4.6)

and

Mf, Mi - A;), i, j = 1, 2, ..., d,

lim E sup I X,(t) - X,(t -) 1’ = 0,

lim E sup I B,(r) - B i t -)I2 = 0,

lim E sup I A:’((r) - A:’(t- 111 = 0,

SUP I Bt(t) - PAX,@), ds I l o ,

I - Q [ I S T h c 1 8 - m [I 1 5 T h r : 1 r - m [ I S T A G

I I; T A <

, ... , d,

(4.7)

Suppose that PX&O)-’ - v E 9(R”). Then (X , } converges in distribution to the solution of the martingale problem for (A, v).

PmKd 1Rt

t : AL’(t) > t supa,Ax) + 1 for some i 1x1 i r

(4.8)

4. DIFFUSION AWROXIMATIONS 355

and set p,, = M,( A q: A T:). Relative compactness of {en} follows as in the proof of Theorem 1.4(b), which in turn implies relative compactness for { X,,( A I : ) } since { B,,( * A T:)} is relatively compact by (4.6). Fix ro > 0 and let {XJ. A ?Lo)} be a convergent subsequence with limit X'". For all but countably many r < r o ,

( x,,( * A T:), T:) - (x'"( . A T'), T'),

where T' = inf { t : I X'"(t)I 2 r} (i.e., for all r < ro such that P(lims+r T' = T'} = 1).

Again as in the proof of Theorem 1.4(b),

M ' O ( I A T,) 3 X'O(t A T') - h(X'O(s)) ds (4.9) SbAF

LA'' and

(4.10)

are martingales, and by 118's formula, Theorem 2.9 of Chapter 5,

(4. I 1)

is a martingale for each/€ CF(lWd). Uniqueness for the martingale problem for ( A , u ) implies uniqueness for the stopped martingale problem for ( A , v , {x: 1x1 < r}). Consequently, if X is a solution of the martingale problem for (A , v) , then X , ( - A r : ) - X ( . A r ' ) for all r such that P(lims-., r s = T'} = 1 (here T' = inf ( t : IX(t ) l 2 r } ) . But T,-+ og as r-- , m (since X has sample paths in

0

Mio(t A ~ ' ) M y ( l AT') - a,,(X'"(s)) ds

f( X'O(t A T')) - G'( X'"(S)) d~ r'

C,[O, a)), so x, * x .

4.2 Corollary Let a, b, and A be as in Theorem 4.1, and suppose the martingale problem for ( A , v ) has a unique solution for each v E 9(Rd). Let p,,(x, r), n = 1, 2,. .., be a transition function on Rd, and set

(4.12) b,,(x) = n I (Y - x)pn(x* dy) I Y - - X I S I

and

(Y XMY - X) 'pn(X . dy)* 1 I y - x l a l (4.13) aAx) = n

Suppose for each r > 0 and E > 0,

(4.14)

(4.15)

356 WVMJANCE PMNUPUS AND DlFFUSlON APPUOXMTIONS

and (4.16)

Let U, be a Markov chain with transition function &t, r) and define X,(t) = Y,([ntJ). If P x(O) - ' is v, then {X,} converges in distribution to the solution of the martingale problem for (A, w).

Proof. Let ti be as in the proof of Theorem 4.1, and let y, = inf { t : 1 X,(t) - X,( f - ) I > I}. Then (4.16) implies P{yn < r: A T} + 0 for each r > 0 and T > 0. Therefore (see Problem 13), we may as well assume pn(x, {y: ] y - x i > 1)) = 0. Let

(4.17)

0

5. STRONG APPROXIMATION THEOREMS

In this section we present, without proof, strong approximation theorems of Kom169, Major, and Tusnidy for sums of independent identically distributed random variables. We obtain as a corollary a result on the approximation of the Poisson process by Brownian motion. To understand the significance of a strong approximation theorem, it may be useful to recall Theorem 1.2 of Chapter 3. This theorem can be restated to say that if p, v E 9(S) and p(p, w ) < E, then there exist a probability space (n, 9, P) and random variables X and Y defined on (f2, 9, P), X with distribution p, Y with distribution w, such that P(d(X, Y) 2 E } s E.

5.1 Theorem Let p E 9(R) satisfy eexp(dx) < 00 for I a I s a. , some a. > 0. Then there exist a sequence of independent identically distributed random variables {C,} with distribution p, a Brownian motion W with m = E[W(l)] = E[CJ and 0' E var (W(1)) = var (<,) (defined on the same sample space), and positive constants C, K, and rl depending only on p, such that

(5.1)

for each n 2 1 and x > 0, where Sk = z-, 4,.

P rnax IS, - W(k)I > C log n + x I 1 s k S r

5. STRONG APIROXIMAnON THEOREMS 357

Proof. See Komlos, Major, and Tusnidy (1975, 1976). 0

5.2 Corollary Let ( t r } and W be as in Theorem 5.1. Set X,(r) = n-'" Ek;', (tk - E l & ] ) and w,(t) = n- ' / ' (W(nt) - E[W(nt) ] ) . (Note that W,(r) is a Brownian motion with mean zero and var (W,(t)) = t var (tl).) Then there exist positive constants C, K, y , and 1, depending only o n b, such that for T T l , n T I , a n d x > O ,

IX,(f) - W,(r)( > Cn 'Iz log n + x

I t follows that there exists a /I > 0 such that for n 2 2, p ( P X , I , PW, I ) s /In'- log n, where p is the Prohorov metric on .9(D,[O, ou)).

Proof. Let C , , K, , and A , be the C, K, and 1 guaranteed by Theorem 5.1 and set C = 2C, . Then defining w(f) = W(f) - t, the left side of (5.2) is bounded by

2 IS, - w ( ~ ) J > c, log cnr] - c , log r +

sup sup I *(A + s) - *(&)I > c, log n + +} . k s n r Osss 1

The second term in (5.3) is bounded by

sup I w ( s ) ( > C , log n + O S S S I

(5.4)

and for any a > 0,

(see Problem 17). Selecting a > 1, so that aC, > 1,(5.2) is bounded by

(5.6) K , exp { -1, ( - C , log T + - 2

for y = (1' C,)V I , 1 = Al/2, and K = K , + K , .

358 INVARIANCE PRINCIPLES AND DIFFUSION APPROXlMATIONS

For a,, > 0, k = I , 2, ..., with ck a, = l , A > 0 to bedetermined,and n z 2,

(5.3) P{d(X , , W& > An - log n )

I I

s P { r e - ' ;!.i IX,(s) - If(@)( dt > An-'" log n

< ~ p { ~ - l e - l s u p I x f l ( s ) - k K(s)ldr>a,An-I / ' togn)

k S S l

5 c P{sup Ix,,(s) - w,,(~)I > ek-1akAn-"2 log n

s x K k Y e x p { -A(ek- 'akA-C) logn)

s n - ' E K k y e x p { - A ( e k - ' u k A - C - A - ' ) l o g 2 } ,

provided ek-'ak A - C - A-' > 0 and the sum is finite. Note the ak and A can be selected so these conditions are satisfied. Finally, select /3 2 A so that

k s s k

k

k

/ In- ' / ' log n bounds the right side for all n 2 2. n

5.3 Corollary Let ,Y E P(R) be infinitely divisible and I ewp(dx) < oc) for I a I s ao, some a. > 0. Then there exists a process X with stationary independent increments, X(1) with distribution I(, a Brownian motion W with the same mean and variance as X, and positive constants C, K, and A depending only on p such that for T 2 1 and x > 0,

(5.8) IX(t) - W(r)I > C log T + x

5.4 Remark Note that if we replace x by x + y log T, then (5.8) becomes

(5.9) P{sup I X ( t ) - W(t)I > (C + y ) log T + x 0 l s r

Proof. Let ( { k } and W be as in Theorem 5.1, (Note that the C, K, and A of the corollary differ from those of the theorem.) Let { X k } be independent processes with stationary independent increments with the distribution of Xk( 1) being p Since the distribution on R" of (Xk(l)} is the same as the distribution of {tk}. by Lemma 5.15 of Chapter 4 we may assume { X k } , [&}, and W are defined on the same sample space and that Xk(l) = €,. Finally define

(5.10) k - I

X(r) = C C, + Xk(r - k + l), k - 1 s t < k, I = I

9. STRONG APPROXIMATION THEOREMS 359

and note that the left side of (5.8) is bounded by

+ P max sup IRk (s ) l> 3 - l ~ log T + 3 - ' x

+ P max sup I CV(s + k ) - W ( k ) ( > 3 - l ~ log T + 3 - ' x

I k S T $ 5 1

{ L S T $ 5 1

where @(t) = W(r) - E[W(t)] and &t) = X(r) - E [ X ( t ) ] . The result follows 0 from (5.1) and Problem 17.

5.5 Corollary Let X and W be as in Corollary 5.3. Then

(5.12)

Proof. Take y = 1 in (5.9). Then

sup I X ( t ) - w(r)I =- (C + I ) tog 2" + .Y

5 K(I - 2-'))-'e-'".

The construction in Corollary 5.5 is best possible in the following sense. 0

5.6 Theorem Suppose X is a process with stationary independent increments, sample paths in DRIO, a), and X ( 0 ) = 0 as., W is a Brownian motion, and

(5.14)

Then X is a Brownian motion with the same mean and variance as W.

Proof. See Bdrtfai (1966).

360 mvmANcE m ~ m s AND DIFFUSION AUROXIMATIONS

6. PROBLEMS

1. (a) Let N be a counting process (i.e., N is right continuous and constant except for jumps of + 1) with N(0) = 0. Suppose that C is a continuous nondecreasing function with C(0) = 0 and that N ( t ) - C(t) is a martingale. Show that N has independent Poisson distributed increments, and that the distribution of N is uniquely determined.

(b) Let {N,} be a sequence of counting processes, with N,(O) = 0, and let A, , n = 1, 2, . . . , be a process with nondecreasing sample paths such that sup, (A&) - A,(t -)) s 1 and N, - A, is a martingale. Let C be a continuous nondecreasing function with C(0) = 0, and suppose for

each c 2 0 that A#)-+ C(c). Show that N, N where N is the process characterized in part (a). Remark. In regard to the condition sup, (A,(t) - A& - )) s 1, consider the following example. Let Y,, Yz , and A be independent processes, Yl and Y2 Poisson with parameter one and A nondecreasing. Let N,(t) = Y,(A(t)) and A&) = nY,(A(r)/n). Then N, - A, is a martingale and A,(t)-t 0 in probability.

P

2. Let W, and W, be independent standard Brownian motions and define

Show that for I 0 I2t and hence

ach t? E Rz, t? X( t ) is Brownian motion with varianc

(6.2)

is a martingale with respect to 9:'' = 4 0 * X(s): s 5 t ) (cf. Theorem 1.2), but that (6.2) is not (in general) an {d~)-martingale.

3. Let N be a Poisson process with parameter 1, and define V( t ) = (- 1)"'". For n = 1, 2, . . . , let

X,(r) = n - I V(s) ds. (6.3) La' Show that

(6.4)

6. PROBLEMS 361

is a martingale and use this fact to show X , 3 W where W is standard Brownian motion.

4. Develop and prove the analogue of the result in Problem 3 in which V is an Ornstein-Uhlenbeck process, that is, V is a diffusion in 08 with generator AS= +a/.'' - bxf', a, b > O , ~ E Cp(5U).

Let t1, t2, . . . be independent and identically distributed with tl, 2 0 a s , E[&J = p 7 0, and var (Ck) = 6' < 00. Let N(r) = max { k : zf3, (, 5 t ) , and define

5.

X, ( t ) = n - (N(nt) - ;). (a) Show that

(6.6)

i s a martingale. (b) Apply Theorem 1.4 to show that X , = z - X , where X is a Brownian

motion with mean zero and variance parameter a'/p3.

6. Let E be the unit sphere in R'. Let p(x, r) be a transition function on E x B(E) SF ;dying

(6.7)

for some p E ( - I , I ) .

N W + I

k = l W ) = 1 € k - (W + 1)P

Yp(x* dY) = P X , X E , s Define Tf(x) = fJ(y)p(x, dy). Suppose there exists v E B(E) such that

for each x E E andJE C(E). Let { Y(k), k = 0. 1. 2, . . .} be a Markov chain with transition function p(x. r) and define

is a martingale, and use Theorem 1.4 to prove X, * X where X is a three-dimensional Brownian motion with mean zero and covariance

(6. I 1)

362 INVARIANCL PRINCIPUS A N D DIFFUSION APPROXIMATIONS

7. For n = 1, 2, ..., let X , be a process with sample paths in Ds[O, ao), and let M, be a process with sample paths in D,[O, a). Suppose that M, is an (#:lj-martingale, and that (X", Mm)*(X, M). Show that if M is {f:}-adapted and for each t 2 0 {M,(r)} is uniformly integrable, then M is an {Sf}-martingale.

8. Verify the identities in (2.1) and (2.2).

9. Let r be a collection of nonnegative random variables, and suppose there is a random variable q such that for each 4 E r, C s as. Show that there is a minimal such q, that is, show that there is a random variable go such that for each t: IS r, C: s qo 8.5. and there exist 4, E r, i = 1, 2, ..., such that qo = sup, ti. Hint: First assume E [ q ] c Q, and consider

(6.12)

10. (a) Let E = (1, 2, . .. , d ) . Let { Y(k), k = 0, I, 2, . ..} be a Markov chain in E with transition matrix P = ((p,,)), that is, P{ Y(k + 1) = j I Y(k) = i } = p,,. Suppose P is irreducible and aperiodic, that is, pk has all elements positive for k sufficiently large. Let 9, = o{Y(&): k s n ) and 9" = a { Y ( k ) : k L n } , and define cp(m) = sup,, ~J~(S"+~] .F , ) . Show that limm-m m - ' log q(m) = a < 0 exists, and characterize a in terms of P.

(b) Let X be an Ornstein-Uhlenbeck process with generator Af= iuf" - b x r , a, b > 0. Suppose PX(O)-' = v is the stationary distribution

for X . Compute JI,. = $o. given by (2.20). Let X be a reflecting Brownian motion on [O, 11 with generator Af= +a"''. Suppose f X ( O ) - ' = m (Lebesgue measure) and compute JI,,l = J I o , ~ given by (2.20).

11. For n = 1,2,. .. , let {t:, k = 1, 2, . . .) be a stationary sequence of random variables with f { t ; = 1) = p,, and Pit," = 0) = 1 - p , . Let .F; = a((;: i 5 k), 49; = a((;: i 2 k), and define pgm) = sup, pp(g:+m19i).

Suppose np,- i. > 0 and max,., P{ (;+, = 1 IS;} -+ 0 as n -+ 00, and define

(c)

P

(6.1 3)

Give conditions on qJ^,(m) that imply N , - N , where N i s a Poisson process with parameter A.

12. Let { k, k E Z} be stationary and define 9: = u(V,: 1 s k 5 n ) and 9" = o( V,: k 2 n). Show that for each m, p,,(S"+'"[gF~) is a nondecreasing function of n and qp(m) = limndm ~J,,(.P"'~ I Pip."), where cpP(m) is given by (3.1).

6. ~OIILEMS 363

13. For n = I , 2, . . ., let p,(x, r) and v,,(x, r) be transition functions on wd x a(Wd). Suppose p,(x, r n B(X, I)) = v.(x. r n E(X, I)), x E uud. r E B(W'), and limn-.m sup, np,(x, E(x, 1)') = 0. Show that for each Y E Sr(Wd) there exist Markov chains { U,(k), k = 0, I , 2, . ..) and {Z,(k), k = 0, I , 2, ...} with PU,(O)-' = PX,(O)-' = u, such that Y, corresponds to p,(x, r) and 2, corresponds to v,(x, r), and for each K > 0,

P{ U,(k) # Z,(k) for some k I n K } = 0.

14. For n = 1, 2, . .. let U, be a Markov chain in En = { k / n : k = 0, I , ..., n} with transition function given by

U,(k + 1) = n I Y,,(k) = X) = (Y)d(l - x)"-' j .

Apply Corollary 4.2 to obtain a diffusion approximation for Y,([nt]) (cf. Chapter 10).

15. Let E = (0, I, 2, . . .), and let 2, be a continuous-time branching process with generator

m

(6.15) A m ) = Ak c p J / ( k + I - 1) -/(&)) 1 = 1

for/€ C,(E), where p, 2 0, I = 0, 1. 2, ..., A > 0, and clmp, p, = 1. Define X,(t) = Z,(nf)/n, and assume PX,(O)- ' 3 v E 9([0, 00)). Suppose CEO Ip, = 1 and CEO I'p, c 00. Apply Theorem 4.1 to obtain a diffusion approximation for X , (cf. Chapter 9).

16. Let N, and N 2 be independent Poisson processes and let F, G E C'(W). Apply Theorem 4.1 to obtain a diffusion approximation for X, satisfying

(6.16) k,(t) = ( - l)N"""'nF(Xn(t)) + (- l)Nlfnz'hG(Xn(l))

(cf. Chapter 12).

Hint: Find the analogue of the martingale defined in (6.4).

17. Let X be a process with stationary independent increments satisfying E[X(t)] - 0 for I 2 0 , and suppose there exists a. > 0 such that &(m' E[&( I 3 < co for la1 s ao. Show that exp (aX( f ) - r$(a)) is a martingale for each a, I a I s ao, and that for 0 < a I; a.

361

7. NOTES

lNVARlANCE PRINCIPLES AND DIFFUSION APPROXIMATIONS

The invariance principle for independent random variables was given by Donsker (1951). For discrete time, the fact that the conditions of Theorem 1.4(b) with A, given by (1.22) imply asymptotic normality was observed by Levy (see Doob (1953, page 383) and developed by Dvoretzky (1972). Brown (1971) gave the corresponding invariance principle. McLeish (1974) gave the discrete-time version of Theorem 1.4(a). Various authors have extended and refined these results, for example Rootzen (1977, 1980) and Gansler and Hausler (1979). Rebolledo (1980) extended the results to continuous time. The version we have presented is not quite the most general. Sce also Hall and Heyde (1980) and the survey article by Helland (1982).

Uniform mixing was introduced by Ibragimov (1959) and strong mixing by Rosenblatt (1956). For p = r = I, (2.8) is due to Volkonskii and Rozanov (1959), and (2.14) is due to Davydov (1968). For = cp,(YI A@), (2.26) appears in Ibragimov (1962). A variety of other mixing conditions are discussed in Withers (1981) and Peligrad (1982).

A vast literature exists on central limit theorems and invariance principles under mixing conditions. See Hall and Heyde (1980), Chapter 5, for a recent survey and Ibragimov and Linnik (1971). Central limit theorems under the hypotheses of Theorems 3.1 and 3.3 (assuming (3.19)) were given by Ibragimov (1962). Weak convergence assuming (3.19) was established by Billingsley ( 1 968). The proof given here is due to Heyde (1974).

Theorem 4.1, in the form given here, is due to Rebolledo (1979). Corollary 4.2 is due to Stroock and Varadhan (1969). See Stroock and Varadhan (1979), page 266. Skorohod (1965), Borovkov (1970), and Kushner (1974) give other approaches to diffusion approximations.

Theorem 5.1 is due to Komlbs, Major, and TusnAdy (1975, 1976). See also Major (1976) and CsBrgti and Riv6sz (1981). Theorem 5.6 is due to Bhrtfai (1966).

The characterization of the Poisson process given in Problem I(a) is due to Watanabe (1964). Various authors have given results along the lines of Problem l(b), Brown (1978), Kabanov, Lipster, and Shiryaev (1980), Grigel- ionis and MikuleviEios (198 I), and Kurtz (1982).

The example in Problem 2 is due to Hardin (1985). There is also a vast literature on central limit theorems and related invari-

ance principles for Markov processes (Problems 3, 4, and 6). The martingale approach to these results has been taken by Maigret (1978), Bhattacharya (1982), and Kurtz(1981b).

The purpose of this chapter is to list conditions under which certain linear operators, corresponding (at least intuitively) to specific Markov processes, generate Feller semigroups or have the property that the associated martingale problem is well-posed. In contrast lo other chapters, here we reference other sources wherever possible.

Section I contains results for nondegenerate diffusions. These include classical, one-dimensional diffusions with local boundary conditions, diffusions in bounded regions with absorption or oblique reflection at the boundary, difh- sions in B8‘ with Holder continuous coefficients, and diffusions in R’ with continuous, time-dependent coeflicients.

Section 2 concerns degenerate diffusions. Results are given for one- dimensional diffusions with smooth coeficients and with Lipschitz continuous, time-dependent coeficients, diffusions in 88’ with smooth diffusion matrix and with diffusion matrix having a Lipschitz continuous, time-dependent square root, and a class of diffusions in a subset of R’ occurring in population genetics.

In Section 3, other processes are considered. Results are included for jump processes with unbounded generators, processes with LCvy generators including independent-increment processes, and two classes of infinite particle systems, namely, spin-flip systems and exclusion processcs.

365



366 EXAMPLES OF GENERATORS

1. NONDECENERATE DIFFUSIONS

We begin with the one-dimensional case, where one can explicitly characterize the generator of the semigroup. Given - 00 5 ro < r , s 00, let 1 be the closed interval in W with endpoints ro and r , , I" its interior, and its closure in [ - 00, 033. In other words,

(1.1) I = [ ro , r , ] n W, I" = ( ro , r,), 7 = [rot rl].

We identify C(i) with the space o f j E C(I") for which limx+,, /(x) exists and is finite for i = 0, 1. Suppose a, b E C(I") and a > 0 on I". Then there is at least one restriction of

d' d G = &x) - + b(x) -

dx2 dx

acting on { f ~ C(i) n C'(I"): Gf E C(i)} that generates a Feller semigroup

To specify the appropriate restrictions, we need to introduce Feller's on c(i).

boundary classification. Fix r E (ro, r I ) and define B, m, p E C(Io) by

Define u and u on by

( 1.6) 4x1 = I ' m dp, W = r~ dm.

Then, for i = 0, 1, the boundary r, is said to be

regular if u(r3 e OD and dr,) < 00,

exit if u(ri) < 00 and dr,) = 00,

entrance if u(r,) = OD and u(r,) < 00,

natural if u(r,) = 00 and V(rJ = 00.

Regular and exit boundaries are said to be accessible; entrance and natural boundaries are termed inaccessible. See Problem 1 for some motivation for this terminology.

(1 -7)

1 NONDECENERATE DIFFWIONS 367

Let

(1.8)

and for i = 0, I , define

( 1.9) 9Ji = 9, r , inaccessible,

9 = (SE C(f) n C2(P) : G ~ E : C(i)}.

(1.10) 9, = 9: lim Gf(x) = o

and

x -?I

I , (1.1 I ) 9, = E 9: q, lim Gf(x) = (- I)'(t - q,) Iim eB(xl/'(x) ,

ri regular,

if x-rt I -n I where qr E [O, I ] and B is given by (1.3).

1.1 Theorem Given -a 5 ro < rl 5 00, define I, I", and i by (1.1). Suppose a, 6 E C(lo) with a > 0 on I", and define G, go, and 9, by (1.2) and (1.8HI.I l), where qr G [0, I ] is fixed if ri is regular, i = 0, I . Then ((5 Gf): f E 9,, A 9,} generates a Feller semigroup on C(T).

Proof. See Mandl (1968) (except for the exit case). 0

1.2 Corollary Suppose, in addition to the hypotheses of Theorem 1.1, that infinite boundaries of I are natural. Then {(J G j ) : / e e(/) n go n a,, G ~ E 4 1 ) ) generates a Feller semigroup on c(/). Proof. See Problem I . 0

1.3 Remark We note a simple, suficient (but not necessary) condition for the extra hypothesis in Corollary 1.2. If there exists a constant K such that

(1.12) a(x) s K(1 + x') , Ib(x)l 5 K(1 + Ixl),

then infinite boundaries of I are natural. The proof is left to the reader (Problem 2). 0

x E I:,

For some applications, it is useful to find a core (such as C:(I)) for the generator in Corollary 1.2. In Section 2 we do this under ccrtain assumptions on the coefficients a and h.

We turn next to the case of a bounded, connected, open set C l c W", where d 2 2. Before stating any results, we need to introduce a certain amount of notation and terminology.

368 EXAMRESOFCENERATORS

Let 0 < p 5 1. A function f E f?(lZ) is said to satisfy a Holder condition with exponent p and Holder constant M if

(1.13)

where the supremum is over 0 < p S po ( p o fixed), x E R', and components I/ of f2 n B(x, p). We denote M by I f I,, .

(1.14) Cmbr(n) = { l e C"(f2): ID"fI,, < og whenever la1 = m},

where Co(Q) = c(n), u e (Z+)d, D" = 8;' - * - d?, and I a I = a1 + * * + ad. Observe that functions in Co*r(fi) netd not have continuous extensions to 5, as such extensions might have to be multivalued on an.

Regarding elements of W' as column vectors, a mapping of Wd onto R' of the form y = U(x - xo), where xo E df2 and U is an orthogonal matrix (U U 5= I,), is said to be a local Cartesian coordinate system with origin at xo if the outer normal to an at xo is mapped onto the nonnegative yd axis. For m = 1, 2, . . . , we say that dR is of class Cm*" if there exists p > 0 such that for every xo e 80, n B(xo, p) is a connected surface that, in terms of the local Cartesian coordinate system (yI , . . . , y,) with origin at xo , is of the form yd = u(yl, . . . , yd- ,), where u E Cmi "@), D being the projection of af2 n B(xo, p) (in the local Cartesian coordinate system) onto yd = 0. Assuming aQ is of class C**#, a function cp: df2+ R is said to belong to Cm*r(af2) if, for every xo E dR, cp as a function of (yl, ..., yd-l) belongs to C"*qD), where the notation is as above. Note that if dn is of class C""' and if 0 $ k s m, then each function in Ckm"(n) has a (unique) continuous extension to a, and its restriction to aQ belongs to C?* "(8Q).

SUP P-yIIuP f(Y) - inf m} = M, Y.V Y r v

For m = 0, 1, . . . , we define

We consider

(1.15)

treating separately the cases of absorption and oblique reflection at the boundary. s d denotes the space of d x d nonnegative-definite matrices.

1.4 Theorem Let d 2 2 and 0 0.

Then, with G defined by (1.1 S), the closure of (1.17)

is single-valued and generates a Feller semigroup on C(n).

1ei-r

A = { (A C j f ) : / ~ C'*"(n), Gf= 0 on an}

1 NONMCENERATE DIFFUSIONS 369

proof. We apply Theorem 2.2 of Chapter 4. Clearly, A satisfies the positive maximum principle. If 1 > 0 and g E C’*”(n), then, by Theorem 3.13 of Ladythenskaya and Ural’tseva (1968), the equation J.f- G f = g has a solution f~ C’*”(Q) with f = 1- ’g on 3s). It follows that Gf= 0 on ds), so IE 9 ( A ) , proving that 9 ( A - A ) =I Cz*”(n) for every A > 0. To show that 9 ( A ) is dense in C(@ let f E C2*”(6). For each 1. > 0, choose

hA E 9 ( A ) such that (A - A)hA = A f - GJ Then, as I - , 00,

(1.18) llf--hAIl = SUP I/(X)--hA(x)l= sup A-’IGY-(x)l-+O. x m a n X 6 8 ( 1

where the first equality is due to the weak maximum principle for elliptic 0 operators (Friedman (1979, Theorem 6.2.2).

1.5 Theorem c: an-, R’, c, E C’”’(8R) for i = 1, ... , d, and

Suppose, in addition to the hypotheses of Theorem 1.4, that

(1.19) inf c(x) * n(x) > 0, x c a o

where n(x) denotes the outward unit normal to dQ at x. Then, with G defined by (1.1 5). the closure of

( 1.20)

is single-valued and generates a Feller semigroup on C(n).

A 3 ((J G f ) : f ~ CZ*”(n), c 9 Vf = 0 on as))

Proof. Again, we apply Theorem 2.2 of Chapter 4. Because c(x) * n(x) # 0 for all x E 3s). A satisfies the positive maximum principle. ( I f f E 9 ( A ) has a positive maximum at x E do, then Vf(x) = 0.) By Theorem 3.3.2 of Ladyzhenskaya and Ural’tseva (1968). there exists 1 > 0 such that for every g E Cz*r(cl) the equation if- Gf= g has a solutionJE C’*”(n) with c . Vf = 0 on as). Thus

It remains to show that W A ) is dense in C(n), or equivalently (by Remark 3.2 of Chapter 1). that the bp-closure of 9 ( A ) contains C(b). By Stroock and Varadhan (1971) there exists for each x E a solution X , of the C,[O, 00)

martingale problem for (A, a,). Consequently, for each f E C(n),

(1.21)

Since the right side of (1.21) belongs to the bp-closure of 9(j), the proof is complete. 0

@(A - A ) 3 c**”(n).

/(x) = bp-lim n e ~ “ ‘ E [ f ( X , ( t ) ) ] dt . r 11-4 m

Let us now consider the case in which fl = R’.

1.C Theorem Let (I: Rd-+ Sd and 6: Rd+ R' be bounded, 0 0, and suppose that

(1.22) 14x1 - &9I + IW - 4 Y ) l s K l x - YI', XS Y E R',

and

(1.23) inf inf 8 - a(x)8 > 0. rebP1 l8I-1

Then, with G defined by (l.lS), the closure of { (A G f ) : f ~ C,.D(Rd)} is single- valued and generates a Feller semigroup on t(iRd).

Proof. According to Theorem 5.1 I of Dynkin (1965), there exists a strongly continuous, positive, contraction semigroup { T(t)} on e(Rd) whose generator A extends G 1c,2(m,)r) (and therefore G It suffrces to show that A is conservative and that C:(R') is a core for A.

Given f E c(Rd) and t > 0, the estimate in part 2" of the proof of Theorem 5.11 of Dynkin (1965), with (0.41) and (0.42) in place of (0.40), implies that d, T(r)f and a, d, T(t)J exist and belong to C(Rd). Thus, T(t): e2(Rd)+ &"(R') for all t 2 0, so e2(Wd) is a core for A by Proposition 3.3 of Chapter 1. Let h E Cj(Rd) satisfy xB,o. ), 5 h s X&o, 3, and approximatefE Cz(W') by (fh,} c C,"(Wd), where h,(x) = h(x/n), to show that Cj(R') is a core for A. Similarly, using the sequence {(h,, Gh,)}, we find that A is conservative. Finally, choose cp E C7(Rd) with cp 2 0 and cp(x) dx = 1, and approximate f E Cj(R') by { f a cp,} c Cp(Wd), where cp,(x) = ndcp(nx), to show that C:(Wd) is a core for A.

D (Note that J * cp, has compact support because bothfand cp,, do.)

If one is satisfied with uniqueness of solutions of the martingale problem, the assumptions of Theorem 1.6 can be weakened considerably. Moreover, time-dependent coeficients are permitted. Consider therefore

I d ' 2 i , ,=Jl I = I

( I .24) G = - c a,~(t, x) a, a, + 1 bi t , x ) a,.

1.7 Theorem Let a: [O, 00) x Rd-r S, and 6: [O, 00) x Wd+ R' be locally bounded and Bore1 measurable. Suppose that, for every x E Wd and ro > 0,

(1.25)

and

(1.26)

inf inf 8 + a(t, x)d > 0 oscrco i e i = i

lim sup I&, y ) - a(t, x)l = 0. y - x o s l s l o

Suppose further that there exists a constant K such that

( I .27) I&, x)l K(l + lxl'), t 2 0, x E Wd,

2. DEGENERATE DIFFUSIONS 371

and

(1.28)

Then, with G defined by (1.24). the martingale problem for ( ( A GI):/€ C:(W”)} is well-posed.

x * qr , x) 5 K( 1 + I x 12). t 2 0, x E Rd.

Proof. Recalling Proposition 3.5 of Chapter 5, the result follows from Theorem 10.2.2 of Stroock and Varadhan (1979) and the discussion following their Corollary 10.1.2. 0

2. DEGENERATE DIFFUSIONS

Again we treat the one-dimensional case separately. We begin by obtaining suficient conditions for Cp(/) to bc a core for the generator in Corollary 1.2.

2.1 Theorem Given -a, < ro < r , I 00, define I and I” by (1.1). Suppose a E C2(/), a 2 0, a“ i s bounded, b: /-+ Iw is Lipschitz continuous (that is,

(2.1)

Then with G defined by (1.2). the closure of { (J G J ) : J E CT(I)} is single-valued and generates a Feller semigroup {T( t ) } on &). If a > 0 on I ” , then { T(t)) coincides with the semigroup defined in terms of a and 6 by Corollary 1.2 (with q, = 0 if ri is regular, i = 0, 1).

SUP,.,,, .. . y l b ( Y ~ - b ( ~ ~ l / I ~ - Y l ~ a )*and dr,) = 0 $ ( - I)%(r,) if I r , I < 00, i = 0, I .

The proof depends on a lemma, which involves the function space (?!!J/), defined form = 0, I , ... and y 2 0 by

(2.2)

where cp,(x) = ( I + x ~ ) ” ~ . Note that (?;(I) = e‘“(I).

CYy(/) = (JE Crn(f): fppy pk’ E QI), k = 0, . . . , m},

2.2 Lemma Assume, in addition to the hypotheses of Theorem 2.1, that b E C2(I ) and b” is bounded. Then there exists a (unique) strongly continuous, positive, contraction semigroup {TO)} on e(/) whose generator A is an extension of { (J G J ) : J E t(/) n C2(f), Gf E e(I)}; moreover:

Proof. This is a special case of a result of Ethier (1978). 0

Proof of Theorem 2.1. c2.,(/) c &‘(I) n Cz(/) n A - ’ e ( l ) , so under the additional assumptions of Lemma 2.2, ct z ( I ) is a core for A by Proposition 3.3 of

372 EXAMPLES OF GENERATOPS

Chapter I. To obtain chis conclusion without the additional assumptions, choose {b,,} c C'(I) such that each b,, satisfies the conditions of Lemma 2.2,

lib,, - bll = 0, and sup,, llbill E M < 00. This can be done via convolutions. For each n, let {q(f)} be associated with a and 6, as in Lemma 2.2. We apply Proposition 3.7 of Chapter 1 with Do = 9 ( A ) = et2(I), D1 = el(I), lll/lll = l l f l l + l l f ' l l , w = M y and en = 116. - bll, concluding that c!,(I) is a core for A under the assumptions of the theoiem. The remainder of the proof that C;(f) is a core for A (and the proof that A is conservative) is analogous to that of Theorem 1.6. For the proof of the second conclusion of the theorem, see Ethier (1978). il

Of course, one can get uniqueness of solutions of the martingale problem under conditions that are much weaker than those of Theorem 2.1. One of the assumptions in Theorem 2.1 that is often too restrictive in applications is the requirement (when b E C1(I)) that b' be bounded, because, in the context of Theorem 1.1, infinite boundaries are often entrance. We permit time- dependent coefficients and so we consider

(2.3) d' d

dx2 dx G = +a(t, X) - + b(t, X) -.

2.3 Theorem Given - 00 s ro < r , s 00, define I by (l.l), and let a and b be real-valued, locally bounded, and Bore1 measurable on [O, 00) x I with a r: 0. Suppose that, for each n 2 1 and to > 0, a and b are Lipschitz continuous in 1x1 s n, uniformly in 0 5 f s to . Suppose further that there exists a constant K such that (1.27) and (1.28) hold with R' replaced by I , and

(2.4)

Then, with G defined by (2.3), the martingale problem for {(A G ~ ) : / E C,""(f)} is well-posed.

a(r, r,) = 0 s (- l)'b(t, r,) if I riI < co t 2 0, i = 0, 1.

Proof. Existence of solutions follows from Theorem 3.10 of Chapter 5, together with (2.4). (Extend a to be zero outside of I and b by setting b(r, x) = b(r, ro), x < ro , and b(r, x) = b(r, r,) , x > r , .) Uniqueness is a consequence of

0 Theorem 3.8, Proposition 3.5, and Corollary 3.4, all from Chapter 5.

Unfortunately, the extent to which Theorem 2.1 can be generalized to d dimensions is unknown. However, results can be obtained in a few special cases.

2.4 Proposition Let u: R'-+ R'B W' and b: R ' d R' satisfy buy 6, E C'(R') for i. j = 1, . . . , d, and put a = 00'. Then, with G defined by (1.1 5), the closure of {(A G / ) : J c CF(8a')) is single-valued and generates a Feller semigroup on e( R').

2. DECLWRATE DIFFUSIONS 373

Proof. A proof has been outlined by Roth (1977). The details are left to the reader (Problem 4). 0

The following result generalizes Proposition 2.4.

2.5 Theorem Let a : Rd-+ & satisfy a,, E C2(Rd) with d, d,a,, bounded for i, j , k, 1 = I , ..., d, and let b: Wd-+ W' be Lipschitz continuous (i.e., S U ~ , . ~ ~ ~ ~ , ~ + ~ IMx) - b(y)l/(x - yl < 03). Then, with G defined by (1.15). the closure of {(f, G f ) : f ~ C,"(Wd)} is single-valued and generates a Feller semigroup on ~ ( I R ~ ) .

Proof. We simply outline the proof, leaving i t to the reader to fil l in a number of details. First, some additional notation is needed. For y 2 0 define cpy : R'-, (0, 00) by cp,(x) = ( I + I x I')"'' and

(2.5)

For rn = I , 2,. . . and y 2 0 define

(2.6) emy(~d) = {/E cm(rwd): V ~ D Y E Q R ~ ) if la1 5 m).

A useful norm on e'!! $Rd) is given by

e_y(lW') = { / € C(Wd): f p , j E Qrw",)

Finally, we define

(2.8) ea-,mc[o, r] x w ~ ) = ( J E C " . ~ ( [ O , 7-1 x r w d ) : qp=f

E C([O, TJ x R ~ ) i f la1 5 m ) .

Suppose, in addition to the hypotheses df the theorem, that hi E C'(Rd) and d k b i , d, d,b, are bounded for i, k, 1 = I , . .., d. Then there exist sequences d"): Rd--t R'@ 08' and b'"): Rd-+ Rd with the following properties, where a'"' = t~'"'(t7'"))~: u:;), 4"' E Cm(Wd), a$) + a,, and by'-, b, uniformly on compact sets, and aj;'/cp, , bf'"/cp, , 8, d,aj;', d, br', 8, d, h]"' are uniformly bounded, i.1, k, 1 - I , ..., d.

Fix n for now. Letting G, be defined in terms of a'"' and 6'"' as in ( I . 1 3 , one can show as in Problem 5(b) that the closure of ( ( A G,$) : /E C:(W')} is single-valued and generates a Feller semigroup { T , ( t ) } on c(Wd). (This also follows from Proposition 2.4.) Moreover, a slight extension of this argument shows that for each m 2 I , T,(t): em(lRd)-+ em(Rd) for all t 2 0 and (q(t)) restricted to Cm(Rd) is strongly continuous in the norm 11.11 C.=fR., (recall (2.7)). A simple argument involving Cronwall's inequality implies that for each y > 0, there exists I , > 0, such that T , ( r ) r p .,, s exp (A,,. , r ) c p . for all r L 0. Using the fact that k'(Rd) n C-JRd) c C!,(Wd) for y sufliciently large, we conclude that ifJe C7(Wd), t > 0, and u,(s, x) _= T,(t - s)/(x), then u, E e".j([O, t ] x Rd) A eo. 4([0, t ] x Rd) and dun/& + G, U, = 0.


We can therefore apply Oleinik's (1966) a priori estimate (or actually a simple extension of it to C'!!,(W')) to conclude that there exists a 2 0, depending (continuously) only on the uniform bounds on a2;)/q2, bj"l/cp,, dk d,at', 8, bj"), and d k d, by), such that

for all fc C:(R"), all n, and all l 2 0. (The proof is essentially as in Stroock and Varadhan (1979), Theorem 3.2.4.) It follows that for each n and t z 0, T,(t): et3(Wd)+ c?3(Iw') and (2.9) holds for all/€ Ct3(R'). Since

(2.10)

En I1 f I1 ~t ,(w)

for all J E C?,(Rd) and all n, where limn-,- E, = 0, the stated conclusion follows from Proposition 3.7 of Chapter 1 with Do = CF(R') and !@A) = D, = e?,(R"), at least under the additional hypotheses noted in the second paragraph of the proof. But these are removed just as in the proof of Theorem 2.1, the analogue of Lemma 2.2 following as in (2.9) from Oleinik's result. 0

To get uniqueness of solutions of the martingale problem in general, we need to assume that a has a Lipschitz continuous square root.

2.6 Theorem Let 6 : 10, 00) x Rd-+ R'@ R' and b: [0, 00) x W ' 4 W' satisfy the conditions of Theorem 3.10 of Chapter 5, and put a = goT. Then, with G defined by (1.24), the martingale problem for {(J G f ) : f ~ CF(Rd)} is well- posed.

Proof. The result is a consequence of Theorems 3.10 and 3.6, Proposition 3.5, 0 and Corollary 3.4, all from Chapter 5.

2.7 Remark Typically, it is a, rather than a, that is given, and with this in mind we give suflicient conditions for all2, the Sd-valued square root of a, to be Lipschitz continuous. Let a: [0, a) x Wd-+S, be Bore1 measurable, and assume either of the following conditions:

(a) There exist C > 0 and 6 > 0 such that (2.1 1)

and In(t, y) - a(t, x)l 5 Cly - X I , t 2 0, x, y E P,

(2.12) inf inf 8 . a(c, x)8 > 6. ( I . XI E 10. m) a I)r lei = 1

2. DECENRATE DIFFUSIONS 375

(b) a,,& a ) E Cz(Rd) for i, j = 1, . . ., d and all t 2 0, and there exists 1 > 0 such that

(2.1 3) sup max sup 18 - (a:aHt, x)O I s 1. f i . s ) e [ O , m ) ~ W I 5 f d d l S l = 1

Then all2 is Bore1 measurable and

(2.14) lal/'(t, y) - a"'(t, x)l 5 K l y - X I , I L 0, X, y E R',

where K = C/(26"') if (a) holds and K = d(2L)"' if (b) holds. See Section 17 5.2 of Stroock and Varadhan (1979).

We conclude this section by considering a special class of generators, which arise in population genetics (see Chapter 10).

2.8 Thcorem Let

(2.15) d

I= I

define a : Kd-+ s d by a,,(x) = x1(dIj - xj), and let h : K d - + Rd be Lipschitz continuous and satisfy

bAx) 2 0 if x E K d and xi = 0, i = 1, ..., d, (2.16) 1 d

bi(x) 5 0 if x E K1 and 1 x , = I . 1 = I I = I

Then, with G defined by (1.15), the closure of { ( J G J ) : f e c 2 ( K d ) } is single- valued and generates a Feller semigroup on C(K,). Moreover, the space of polynomials on K, is a core for the generator.

The proof is quite similar to that of Theorem 2.1. It depends on the following lemma from Ethier (1976).

2.9 Ltmma Assume, in addition to the hypotheses of Theorem 2.8, that hl , . . . , bd E c4(&). Then the first conclusion of the theorem holds, as do the following two assertions:

I , 2.

/ E C ' ( K ~ ) and r 20,

376 EXAMRES OF GENERATORS

Proof of Theorem 2.8. Choose b'"): Kd- , w' for n = 1, 2, ... satisfying the conditions of Lemma 2.9 such that limn-- Jib'") - 611 I= Oand

(2.18)

The latter two conditions follow using convolutions. To get (2.16), it may be necessary to add &,(I - (d + l)x,) to the bj"'(x) thus obtained, where 6 . 3 O+. The first conclusion now follows from Proposition 3.7 of Chapter 1 in the same way that Theorem 2.1 did. The second conclusion is a consequence of the fact that the space of polyiiomials on K d is dense in C2(K1) with respect to the norm

(2.19)

(see Appendix 7). 0

3. OTHER PROCESSES

We begin by considering jump Markov processes in a locally compact, szpar- able metric space E, the generators for which have the form

where R E B,,,(E) is nonnegative and p(x, r) is a transition function on E x @E). We assume, among other things, that 1 and the mapping x-+ p(x, .) are continuous on E. Thus, if E is compact, then A is a bounded linear operator on C(E) and generates a Feller semigroup on C(E). We can therefore assume without further loss of generality that E is noncompact. The case in which E = {0, 1, . . .} is treated separately as a corollary.

3.1 Theorem Let E be a locally compact, noncompact, separable metric space and let E" = E u {A} be its one-point compactification. Let 1 E C(E) be nonnegative and let ~ ( x , r) be a transition function on E x g ( E ) such that the mapping x-+p(x, .) of E into P(E) is continuous. Let y and q be positive functions in C(E) such that l / y and l/q belong to C ( E ) and

(3.2)

3. OTHER PROCESSES 377

lim A(x)p(.x, K) = 0 for every compact K c E, x - A

(3.3)

(3.4)

(3.5)

Then, with A defined by (3.1). the closure of ( ( A A S ) : / € C ( E ) , ~ J ' E C(€), t?((E)} is single-valued and generates a Feller semigroup on t?((E). More-

over, C,(€) is a core for this generator.

Proof. Consider A as a linear operator on C(E) with domain % ( A ) = { f ~ C((E): y f ~ c('(E)) c e ( E ) . To see that A : 9 ( A ) --, C(E), let J E .$@(A) and observe that AJE C(E) and

(3.6)

by (3.2) and (3.4). Using the idea of Lemma 2. I of Chapter 4, we also find that A is dissipative.

We claim that #(a - A ) =I 9 ( A ) for all r sufliciently large. Given n 2 1, define A, on C(E) as in (3.1) but with A(x) replaced by ).(x)An. By (3.3), A,: Cc(€)-+ €(€), and hence A,, is a bounded linear operator on C ( E ) satisfying the positive maximum principle. i t therefore generates a strongly continuous. positive, contraction semigroup { K(f)(t)) on c(E) . By (3.4), there exists w 2 0 not depending on n such that yA,,( I/y) I w, so

(3.7)

for all r 2 0. Let16 9 ( A ) . By (3.7), K ( t ) f ~ 9(A)and

378 EXAMFlES Of C M R A T O R S

so

by (3.2). Since /and n were arbitrary, we conclude from Lemma 3.6 of Chapter 1 that W ( 1 - A ) 3 9 ( A ) .

Thus, by Theorem 4.3 of Chapter 1,

is single-valued and generates a strongly continuous, positive, contraction semigroup {T(t)) on e((E). Clearly, iffe 9(Ao), then Aofis given by the right side of (3.1) and A,f- A. f as n-, 00. It follows from Theorem 6.1 of Chapter 1 that T,(t)f-+ T(r)f for all f e c ( E ) and r L 0. In particular, by (3.7), T(t)(l/y) s e""(l/y) for all I 2 0, so T(t): 9 ( A ) - 9 ( A ) for every I 2 0. We conclude from Proposition 3.3 of Chapter 1 that 9 ( A , ) n 9 ( A ) is a core for A o , that is, the closure of {(S, A f ) : f e e(€), yfe C(E), A f e e ( E ) } generates {TO)}.

Let /E W A , ) n W A ) and choose {h,} c CJE) such that x ~ ~ . ~ ll(r)l, s h, 5; I for each n, and observe that {fh,} c CJE), /h,-./ uniformly, and A ( f h n ) 4 A j boundedly (by (3.7)) and pointwise. Recalling Remark 3.2 of Chapter 1, this implies that Cc(E) is a core for A o .

It remains to show that A. is conservative. Fix x E: E, and let X be a Markov process corresponding to {TA(t)} (see Lemma 2.3 of Chapter 4) with sample paths in DEa[O, 00) and initial distribution 6,. Extend q from E to EA by setting q(A) = 00. Let n 2 1 and define

(3.12) T , = inf { t 2 0: q(X(t)) > n).

Then, approximating q monotonically from below by functions in Cc(E), we find that

3. OTHER mocEssEs 379

for all I 2 0 by (3.3, so E [ q ( X ( t A T,))] is bounded in I on bounded intervals. By the first inequality in (3.13),

(3.14)

and thus

(3.15) nP{ r, I t } 5 E[rl(X(t A ?.))I s q(x)ec3‘

for all t 2 0 by Gronwall’s inequality. I t follows that 7“ = 00 a s and hence X has almost all sample paths in DEIO, a). By Corollary 2.8 of Chapter

0 4, we conclude that A, is conservative.

3.2 Corollary Let E = (0, 1, ...} and

(3.16)

where the matrix (qi,),,,zo has nonnegative off-diagonal entries and row sums equal to zero. Assume also

(3.17)

(3.18)

(3.19)

(3.20)

Then the closure of { (J A f ) : l ~ C,(E)} is single-valued and generates a Feller semigroup on C(E).

Proof. Apply Theorem 3.1 with A(i)p(i, { j } ) = 4ij for i # j , p(i, ( i ) ) = 0, and y ( i ) = q(i) = i + 1. 0

We next state a uniqueness result for processes in W d with LCvy generators, that is, generators of the form

1 ” d

(3.21) GS(x) = - 1 ai,(t. X) 3, 3,SCx) + C 6Alv X) a,S(x) 2 L J = l I = I

3.3 Theorem Let a: [O, 00) x Rd- + S, be bounded, continuous, and positive- definite-valued. 6: LO, 00) x &Id- Rd bounded and Bore1 measurable, and

p: [O, 00) x R'-r A ( R d ) such that fr Iy('(1 + ( ~ i ~ ) - ~ p ( t , x; dy) is bounded and continuous in (t, x) for every r E a(&!'). Then, with G defined by (3.21), the martingale problem for { (A G f ) : f ~ C:(R")} is well-posed.

Proof. By Corollary 3.7 and Theorem 3.8, both from Chapter 4, every solution of the martingale problem has a modification with sample paths in

0 Du[O, 00). The result therefore follows from Stroock (1975).

When a, b, and p in (3.22) are independent of (t, x), A becomes I ' 4

G f ( 4 = - c aij 4 a j m + c 4 W ( X ) 2 1 . j - I 1- I

(3.22)

Every stochastically continuous process in Rd with homogeneous, independent increments has a generator of this form, where a E S,, b E W', fi E A(R"), and IR, I y Iz(l + I y 1')- 'p(dy) < 00 (see Gihman and Skorohod (1969), Theorem V1.3.3). In this case we can strengthen the conclusion of Theorem 3.3.

3.4 Theorem Let u € S d , b e R', and p€Yn(R'), and assume that JRc I yl'(1 + IyI2)-'Cc(dy) < a. Then, with G defined by (3.22), the closure of {(f, Gf):fe e2(R")} is single-valued and generates a Feller semigroup on e(Rd). Moreover, C:(Wd) is a core for this generator.

Proof. If a is positive definite, then by Theorem 3.3, the martingale problem for {(f, G f ) : f ~ C.?(Rd)} is well-posed. For each x E Rd, denote by X x a solution with initial distribution b,, and note that since (Cf)" = G(P) for all f e C:(Wd), where/"(y) r f ( x + y), we have

(3.23) QY(Xx(0)l = + X00))l

for all/€ B(E) and t 2 0. It follows that we can define a strongly continuous, positive, contraction semigroup { T(t)} on C(R9 by letting T(t)f(x) be given by (3.23). Denoting the generator of {T(t)} by A, we have {(A G f ) : f s CF(W')) c A, hence {(I; GI):/€ e2(Rd)} c A. Moreover, by (3.23), T(r): cao(Rd)+ Cm(R') for all t z 0, so em(R') is a core for A by Proposition 3.3 of Chapter 1.

5 h S x ~ , ~ , 2,, and approximatefe Cm(R') by {fh,} c C,"(Rd), where h,(x) = h(x/n), to show that C,"(R") is a core for A. (To check that bp-lim,,, A(Jh,) = Afi it sunices to split the integral in (3.22) into two parts, lyl 5 1 and j y l > 1.) Similarly, using {(ha, Ah")}, we find that A is conservative. The case in which a is only nonnegative definite can be handled

0

Let h E C:(Wd) satisfy x ~ ~ ~ ,

by approximating u by a + &I, e > 0.

3. OTHER PROCESSES 381

We conclude this section with two results from the area of infinite particle systems. The first concerns spin-flip systems and the second exclusion processes. For further background on these processes, see Liggett (1977, 1985).

3.5 Theorem Let S be a countable set, and give { - I , I } the discrete topology and E = { - I , I}’ the product topology. For each i E S, define the difference operator A, on C(E) by A, / ( q ) = /(, q ) - / ( q ) , where (, q)j = ( I - 2 ~ 5 , ~ ) ~ ~ for all j E S. For each i E S, let ci E C(E) be nonnegative, and assume that

(3.24) SUP llcill < GO, SUP C I IA jC i I l C GO. i r S i € S j e s

Then, with

(3.25)

the closure of { (J As): f E C(E), ci II Ai f I\ < a} is single-valued and generates a Feller semigroup on C(E). Moreover, the space of (continuous) functions on E depending on only finitely many coordinates is a core for this generator.

Proof. The first assertion is essentially a special case of a more general result 0 of Liggett (1972). The second is left to the reader (Problem 8).

3.6 Theorem Let S be a countable set, and give { O , 1) the discrete topology and E 5 (0, I}‘ the product topology. For each i , j E S, define the difference operator ArJ on C(E) by 4, f ( v ) =S(,, r t ) - S(rt), where

(3.26)

For each i , j E S, let c,, E C(E) be nonnegative and Y,, be a nonnegative number, and assume that ci, = 0, ci, I c f i , cil I y r j , and yi, = y j f for all i, j E S,

(3.27)

and

(3.28)

where ( k y) I = dkl + ( I - 26kl)vI for all I E S and K is a constant. Then, with

C SUP I c,,h v ) - c,i(v)t < KY,,, i , j E S, L s S w e €

(3.29)

the closure of {(J A/ ) : /E C(E), ~ f , , F S y,,IIAi,jlI < GO} is single-valued and generates a Feller semigroup on C(E). Moreover, the space of (continuous) functions on E depending on only finitely many coordinates is a core for this generator.

382 MAMNEJ OF GENERATORS

Proof. The references given for the preceding theorem apply here. 0

4. PROBLEMS

1.

2.

3.

4.

For each x E I - [ r , , r , ] , let P , E 9(C,[O, 00)) be the distribution of the diffusion process in f with initial distribution Sx corresponding to the semigroup of Theorem 1.1. Let X be the coordinate process on C,[O, a), and define T,, = inf { t 2 0: X ( t ) = y } for y E f. (a) Show that rl is accessible if and only if there exist x E I" and t > 0

such that

(4.1) inf P x { ~ y s r } > 0.

(b) Suppose rl is inaccessible. Show that r l is entrance if and only if there exist y B I" and t > 0 such that

(4.2) inf P,{ry s t } > 0.

Y 6 ( X e l l )

1 6 0. r i )

(c) Prove Corollary 1.2.

Suppose, in addition to the hypotheses of Theorem 1.1, that there exists a constant K such that (1.12) holds. Show that infinite boundaries of I are natural.

Use Proposition 3.4 of Chapter 1 to establish Theorem 2.1 in the special case in which I = [0, 00) and

(4.3) a(x) = ax, b(x) = bx, x E I,

where 0 < a < 00 and -a < b < 00. (The resulting diffusion occurs in Chapter 9.) Hint:

Assume the hypotheses of Proposition 2.4, and for each t 2 0 define the linear contraction S(t) on C(Wd) by

(4.4)

where 2 is N(0, I,,). ( {S ( t ) } is not necessarily a semigroup.) Given t 2 0 and a partition B = (0 = f , ii I , s 5 t, = c) of [O, r ] , define p(n) = max,.,,, ( tr - t,-dand (4.5) S, = S(r, - r , , - , ) S(r, - to) ,

Look for solutions of the form u(r, x) = e-Af')x.

wm = Q l ( x + &x,z + tb(x))I,

4. ruoBiEMs 383

and note that S,: ez(8a’)-+ cz(Rd). Define the norm 111~111 on e’(Wd) by

Prove Proposition 2.4 by verifying each of the following assertions: (a) There exists K > 0 such that

(4.7)

for all/€ e2(Rd) and s, t E 10, I]. (b) There exists K > 0 such that

(4.8) 111 S(t)/III 5 (1 + Kt) IIISIII

for all/€ cz(Rd) and 0 s t 5 1. By parts (a) and (b), there exists K > 0 such that

(4.9)

for allfc c2(Wd)), 0 -< t 5 1, and partitions 7 1 , . nz of [O, t ] .

{pn} c CcP)(R’) by cpn(x) = ndrp(nx). Then there exists K > 0 such that

(c)

IlS,,S- S.,fII 5 K t J m IIIJIII

(d) Choose cp E C:(Rd) with cp 2 0 and cp(x)dx = 1, and define

for allJE e*(Rd)), 0 I f I I, and n. By parts (bHd), for each/E c((Rd) and I 2 0. (e)

(4. I I )

exists and defines a Feller semigroup { T(r)} on @Rd) whose generator is the closure of {(J G / ) : J E C~(W’)}, where G is given by (1.1s).

5. (a) Use Corollary 3.8 of Chapter I to prove the following result. Let E be a closed convex set in W’ with nonernpty interior, let

a : E - r S, and b: E- . R’ be bounded and continuous, and for every x E E let ((x) be an Rd-valued random variable with mean vector 0 and covariance matrix a(x). Suppose that E[ I ((x) Is] is bounded in x and that, for some to > 0,

(4.12) x + J;~ (x ) + tqx) E E as.

whenever x E E and 0 s t 5 t o . Suppose further that, for 0 s I s t o , the equation

381 EXAMtLES OF GENERATORS

(4.13)

defines a linear contraction S(t) on c ( E ) that maps e 3 ( E ) into (?(EX and that there exists K > 0 and a norm 111 - 111 on c 3 ( E ) with respect to which it is a Banach space such that

W S ( 4 = ECS@ + 4 4 x 1 + t W ) I

(4.14)

for allfE e 3 ( E ) and 0 s t s t o . Then, with G defined by (1.1% the closure of {(f, G f ) : f ~ C,"(E)} is single-valued and generates a Feller semigroup on QE).

(b) Use part (a) to prove Proposition 2.4 under the additional assumption thatalj,biE C3(Rd)fori,j= 1, ..., d.

(c) Use part (a) to prove Theorem 2.8 under the additional assumption that bl , . . , , bd E C'(K,).

(d) Use part (a) to prove Theorem 2.1 under the additional assumptions that - 00 < ro < rl < 00, u, b E C3(I), and a = sou,, where ui E C3(I), ar(r,) = 0, and a, > 0 on I" for 1 = 0, 1, and a,,/(ao + uI) is

nondecreasing on I" and extends to an element of C3(f).

6. Fix integers r, s 2 2 and index the coordinates of elements of UP- by

(4.15)

Fix y 2 0 and, using the notation (2.15), define G: C2(K,_ by

111 W S 111 ( 1 + KO 111 f 111

J = { ( i , j ) : i = 1, ..., r, j = 1, ..., s, (i, j ) # ( r , s)).

C(Kr,-l)

where xi. =Eel xfj, xGj = zI1 xu, and x, = 1 - Grij,*, x,,. It follows from Theorem 2.8 that the closure of' {(f, Gf): SE C (K,s- is single- valued and generates a Feller semigroup on C(K,,- Use Proposition 3.5 of Chapter 1 to give a direct proof of this result. Hinr : Make the change of variables

(4.17) pi = Xi., q j = x.j. Dij = Xij - x~ .x . / ,

where i = 1,. . ., r - 1 and j e 1, . . ., s - 1, and define

(4.18)

Let L, be the space of polynomials of "degree " less than or equal to n.

5. NOTES 385

7. Let S be a countable set, give [0, IIS the product topology, and define

(4.19)

Suppose the matrix (q,,)i. sums equal to zero, and

has nonnegative off-diagonal entries and row

(4.20)

Show that, with

1 G = - C xi(6ij - XI) ai aj + C C ' ~ j i x j

(4.21) 2 1. j c s i e S LeS ) the closure of { ( J G ' ) : ~ E C(K), depends on only finitely many coordinates and is twice continuously differentiable] is single-valued and generates a Feller semigroup on C(K).

8. Use Problem 8 of Chapter I to prove Theorems 3.5 and 3.6.

5. NOTES

Theorem 1.1 is a very special case of Feller's (1952) theory of one-dimensional diffusions. (Our treatment follows Mandl (I968).) Theorems 1.4. 1.5, and 1.6 are based, respectively, on partial differential equation results of Schauder (1934), Fiorenza (1959). and Il'in, Kalashnikov, and Oleinik (1962). The first two of these results are presented in Ladyzhenskaya and Ural'tseva (1968) and the latter in Dynkin (1965). Theorem 1.7 is due to Stroock and Varadhan ( 1979).

Essentially Theorem 2.1 appears in Ethier (1978). Theorem 2.3 is due primarily to Yamada and Watanabe (1971). Roth (1977) is responsible for Propo- sition 2.4, while Theorem 2.5 is based on Oleinik (1966). Remark 2.7 is due to Freidlin (1968) and Phillips and Sarason (1968). Theorem 2.8 is a slight improvement of a result of Ethier (1976).

Theorem 3.3 was obtained by Stroock (1975), and Theorems 3.5 and 3.6 by Liggett (1972).

Problem 4 is Roth's (1977) proof. Problem 5(c) generalizes Norman (1971) and Problem 5(d) is due to Norman (1972). Problem 6 can be traced to Littler (1972) and Serant and Villard (1972). Problem 7 is due to Ethier (1981).

9

Because of their independence properties, branching processes provide a rich source of weak convergence results. Here we give four examples. Section I considers the classical Galton-Watson process and the Feller diffusion approximation. Section 2 gives an analogous result for two-type branching models, and Section 3 does the same for a branching process in random environments. In Section 4 conditions are given for a sequence of branching Markov processes to converge to a measure-valued process.

BRANCHING PROCESSES

1. GALTON-WATSON PROCESSES

In this section we consider approximations for the Galton-Watson branching process, which can be described as follows: Let independent, nonnegative integer-valued random variables Zo , &; , k, n = 1, 2, . . . , be given, and assume the {; are identically distributed. Define Z, , 2, , . . . recursively by

Then 2, gives the number of particles in the nth generation of a population in which individual particles reproduce independently and in the same manner. The distribution of { is called the oflspring distribution, and that of 2, the 366



1. GALTON-WATSON PROCESSES 387

initial distribution. We are interested in approximations of this process when Zo is large. The first such approximations are given by the law of large numbers and the central limit theorem.

1.1 Theorem Let Z , , &; be as above and assume Ere;] = m < 00. Then

a s . Z n

Z0-m 20 lim -= m" ( 1 4

In addition let var (&;) 5: E[(&; - m)'] E.E u2 < 00. Then as Z0-+ 00 the joint distributions of

(1.3) W, = Z o 1'2(2n - m"Zo)

converge to those of

where the V, are independent normal random variables with ECV,] = 0 and var ( u ) = u2.

1.2 Remark Note that

(1.5) W: = mW;-, + m'n-1)'* 4. 0

Proof. The limit in (1.2) is obtained by induction. The law of large numbers gives

( 1 4

and assuming limzo,, Z, - ] / Z , = m" ' as.,

2 0

2 0 - m zO 20-m h = l

2, iim - = lim 2, C {: = m as..

we see from (1.8) that it is enough to show the random variables

388 BRANCHING P I O U S S E S

converge in distribution to independent N(0, a') random variables. Let .Fm = o(Z,, I$: I 5 I s n, I 5 k .= 00). Then as in the usual proof of the central limit theoretn

(1.10) Iim ECexp {iw,} I.F,-,I 20-m

= Iim Ec[exp {iOZ;1[2({ - m))lZJ-* = exp { -+a202} as., 21-t-m

where the expectation on the right is with respect to <. Therefore

lim E n exp {ie, v,} = fl exp { -+u26;} (1.11)

and the convergence in distribution follows from the convergence of the char- zo-.a, " , = I 1 ,:I

acteristic functions. 0

Of more interest is the Feller diffusion approximation.

1.3 Theorem Let E[(;] = 1 (the critical case) and let var ( { D = uz < 00. Lei Zp' be a sequence of Galton-Watson processes defined as in (l.l), and suppose Zh"'/m converges in distribution. Then W, defined by

(1.12)

converges in distribution in Dlo, m,[O, a) to a diffusion with generator

(1.13) AS(X) = +&S"(x), / € c,"<co, 00)).

Proof. By Theorem 2.1 of Chapter 8, C:([O, 00)) is a core for A. Note that Z!,")/rn is a Markov chain with values in E, = {l /m: 1 = 0. 1, 2, . . .}, and define

(1.14)

1. GALTON-WATSON ROCESSES 389

where S, = n - [0, c] . Since

c;=, (tr - 1). Suppose the support of 1 is contained in

(1.17)

the integrand on the right of (1.16) is zero if u 5 I - c/x. Consequently,

s 1’ = xlll”ll((c/x) A 1) ’SZ .

S:,xCI - ~)2Il f” l l do O V ( 1 - c / x )

To show that lim,,,-m sup, E,(x) = 0, i t is enough to show limm-m E,(x,) = 0 for every convergent sequence x,,,, where we allow limm-.m x, = 00. Since €[Six] = 0’ for all m, x, inequality (1.18) implies limm-.m E,(x,) = 0 if either limm-m x, = 0 or limm.+m x, = 00. Therefore we need only consider the case xm+ x, for 0 < x < 00. Replacing x by x, in (l . l8), S,,, 3 V , where V is N ( 0 , d), and hence the left side of (1.18) converges in probability to zero and the right side converges in distribution to xll/”ll((c/x) A I)’ V z . Since

the dominated convergence theorem implies limm-.m E,(x,) = 0. 0

Let & be a sequence of nonnegative integer-valued, independent, identically distributed random variables. Given Z o , independent of the ( k , define

I

( I .20)

and for n > 0,

(1.21)

Let n- I

(1.22)

Since

Y” = c z,. 1=0

390 BRANCHING PROCESSES

we see that 2, is a Galton-Watson branching process (i.e., the joint distribution of the 2, defined by (1.21) is the same as in (1.1) for the same offspring and initial distributions). We use the representation in (1.21) and Theorem 1.5 of Chapter 6 to give a gerieralization of Theorem 1.3.

Let ZLm) be a sequence of branching processes represented as in (1.21) with offspring distributions that may depend on m. Let c, > 0 be a sequence of constants with lim,-a c, = co and define

(1.24)

1.4 Theorem Suppose (for simplicity) that Y,(O) is a constant, that Y,(O) = Y(0) > 0, and that { Ym(l)) converges in distribution. Then

Y, 3 Y where Y is a process with independent increments, and W, =+ W in Dl0. mr[O, 00) where W satisfies

(1.27)

for t < T, E lirn,,+m inf {s: W(s) > n} and W(r) = 00 for t 2 t,.

1.5 Remark If W(r) -c 00 for all t 2 0 a.s., then W, - Win D,o, ,,[O, a). 0

Proof. For simplicity we treat only the case where Q = sup, E[ I Y,( 1) I J < 00, which in particular implies E[W,(r)] S Y,(O)e" and hence t, = 00.

Let X,(r) = Y,(t) - Y,(O). Since { Ym(l)} converges in distribution and Ym(0)+ Y(O), {X,(l)} converges in distribution, and we have (1.28) lim E[exp (iOX,(m - 'c; I)} Jmcm = lim E[exp { iOX,( l)}]

n - m m- m

= $(e). It follows that ( I .29) Jim ECexp ( ieX, ( t ) / ] = lim ECexp { iOX,(m- Ic; '))]lmmrl

rn-m m-m

= $40)'.

The independence of the t k implies the finite-dimensional distributions of

1. GALTON-WATSON PROCESSES 391

X , converge to those of the process X having independent increments and satisfying E[P"~"' ] = Jl(0)'. To see that the sequence ( X , } is relatively compact, define 9:'"' = a(X,(s): s s t ) and note that the independence implies

(1.30) E [ ( X , ( C + U ) - X , ( ~ ) J A I ( ~ ) ~ ' ]

S E [ ( X , ( u ) ( A l ] V E X , u + - A 1 , [I ( 91 1 and the relative compactness follows from Theorem 8.6 of Chapter 3.

Under the assumption that sup,,, E[ I Y,( 1) I ] -= 00, the theorem follows from Theorem 1.5 of Chapter 6 if we verify that to = r l as. (defined as in (1.6) and (1.9). both of Chapter 6).

Note that X,,,(t) - X J t - ) ;r - c i l and it follows from Theorem 10.2 of Chapter 3 that M ( t ) E infs,, X ( s ) is continuous (see Problem 26 of Chapter 3). Consequently X(r,) = 0 if ro < 03, and by the strong Markov property the

0 following lemma implies to = rI a.s. and completes the proof.

1.6 lemma Let X be a right continuous process with stationary, independent increments (in particular, the distribution of X(r + u) - X ( r ) does not depend on u) and let X ( 0 ) = 0. Suppose inf,,, X ( s ) is continuous. Then

(1.31)

for all E > 0.

Proof. The process X may be written as a sum X = X I + X 2 where X , and X z are independent, X z is a compound Poisson process, and X , has finite expectation. Then (1.3 I ) is equivalent to

(1.32)

for all E > 0. which in turn is equivalent to showing

(1.33)

for all I 2 0. Let Z(I ) = Xl(r(t)). Then (cf. (1.4) of Chapter 6)

(1.34) r ( t ) = V Z(s) ds = P(s) ds 0

392 N RANCHING nocum

since 2 is nonnegative by the continuity of inf,,, X(s). Let c = E [ X , ( l ) ] . Since r(r) is a stopping time and X,(r ) - cr is a martingale, we have

fI fJ (1.35) E [ t ( t ) ] = E[Z(s)] ds = EIXl(T(S))] ds I Jo

and i t follows that E[r(r)] = 0. Hence we have (1.33). 0

2. TWO-TYPE MARKOV BRANCHING PROCESSES

We alter the model considered in Section 1 in two ways. First, we assume that there are two types of particles (designated type 1 and type 2) with each particle capable, at death, of producing particles not only of its type but of the other type as well. Second, we assume each particle lives a random length of time that has an exponential distribution with parameter A,, i = 1, 2, depending on the type of the particle. Let pLl be the probability that at death a particle of type i produces k offspring of type 1 and I offspring of type 2.

The generator for the two-type process then has the form

+ A 2 2 2 1 p m z , + k, 22 - 1 + 0 - j l z 1 , zz)). k . I

Let (y, , yz) have joint distribution p i , and let (I), , $2) have joint distribution pi,. Assume that Ecy:] c 00 and El#:] < CQ, i = I, 2. Let mtl = ECy,] and mZJ = E[#J We assume m,j > 0 for all i, j ; that is, there is positive probability of offspring of each type regardless of the type of the parent. We also assume that the process is critical; in other words the matrix

has largest eigenvalue 1. This implies that the matrix

(2.3)

has eigenvalues 0 and - q for some q > 0. Let ( v , , w2)T denote the eigenvector corresponding to 0 and (p, , pz)' the eigenvector corresponding to -q . We can take w , , w2 > 0 and pl and p2 will have opposite signs.

1. TWO-TYPE MARKOV ORANCHING PROCESSES 393

Let ( 2 , * 2,) be fixed. We consider a sequence of processes {(Z',"! 2s"')) with generator B and (Zy)(O), Z'f'(0)) = ([nz, J, [nzzJ). Define

and

Then X, and ten" ' are martingales (Problem 8). Since for t > 0, enW' - + 00 as n 4 00, the fact that Ynenn' is a martingale suggests that Y,(t)--+ 0 and, consequently, that Z$"(nt)/n 5 -pl p i 'Z(,"'(nt)/n so that Z',")(nf)/n z p2 X,(t) / ( v I p2 - v,gI) and Z(;'(nt)/n 2: p1 X, ( t ) / ( v , pl - w I p2). This is indeed the case, so the limiting behavior of {X,) gives the limiting behavior of (Z(,")(nr)/n. Zy'(nr)/n) for t > 0.

We describe the limiting behavior of { K] in terms of

which is also a martingale. Define

2.1 Theorem (a) The sequence {(X,, W,)} converges in distribution to a diffusion process (X, W ) with generator

where ail = ( A l p2 a:, - 1, PI a$/(v~ ~2 - P I V Z ) .

I K ( t ) - Y,(O)e ""'I converges lo zero in probability, and hence for 0 < t I < r 2 , l:; nqY,,(s) d.5 converges in distribution to

(b) For T > 0, supr

W , ) - Wt,) . (c) For T > 0.

converges to zero in probability.

394 UANCIU~~~~OCESSES

Proof. It is not difficult to show that C?([O, a) x (-00, 00)) is a core for A (see Problem 3). With reference to Corollary 8.6 of Chapter 4, let f E C:([O, a) x (- 00, 00)) and set

To find 9. so that (f., 9.) E .d, we calculate lim,*o e-'E[f.((t -1- E ) -&)I 93 and obtain

(2.10)

2. TWO-TWE MARKOV BRANCHING mocLssEs 395

and calculating g,, so that (f,, , g,,) E d,,, we obtain

Since A',, is a martingale and

= Xf(0) + r(1, v ; I + A , v ; *)X,(O),

so part (a) follows by Corollary 8.6 of Chapter 4. Observe that (2.6) can be inverted to give

(2.18) Y,(t) - e-""IY,(O) = nqe-"~('-''(W,(t) - W,(s)) ds

+ e-""'(Wn(c) - Ws(0)).

Let U,,(s) = sup, I Ws(t + s) - Wit ) 1. Then for t s T,

Part (b) follows from the fact that Urn* U (U(s) = SUP,,, I W(t + s) - W(t ) l ) and lims-.o U(s) = 0. Part (c) follows from part (b) and the definitions of X, and K. 0

3. BRANCHING PROCESSES IN RANDOM ENVIRONMENTS

We now consider continuous-time processes in which the splitting intensities are themselves stochastic processes. Let X be the population size and Ak, k 2 0, be nonnegative stochastic processes. We want

that is (essentially), we want A&) At to be the probability that a given particle dies and is replaced by k particles. The simplest way to construct such a process is to take independent standard Poisson processes 4 , independent of the A,, and solve

We assume that k yo A&) ds < co as. for all t > 0 to assure that a solution of (3.2) exists for all time. In fact, we take (3.2) to define X rather than (3.1). We leave the verification that (3.1) holds for X satisfying (3.2) as a problem (Problem 4).

3. ORANCHINC PROCESSES IN RANDOM ENVlRONMfNTS 397

By analogy with the results of Sections I and 2, we consider a sequence of processes X, with corresponding intensity processes AP' and define Z,(t) = X,(nr)/n. Assuming X,(O) = n and defining A&) = cp=o (k - l)Ap'(r), we get

m

k - 0 (3.3)

Set

(3.4)

Then

(3.5)

B,(r) = l n A , , ( n s ) ds.

Note that U, i s (at least) a local martingale. However, since B, i s continuous and U, has bounded variation, no special definition of the stochastic integral is needed.

3.1 Theorem *(I?, D) and that there exist a, satisfying a,/n --+ 0 and

(3.6)

Let D,(r) = ro cF=o ( k - l)*A:"'(ns) ds. Suppose that ( B , , D,)

lim ( k - l)'A:"'(ns) ds = 0 as. n-.m 0 &>a.

for al l r > 0. Then 2, converges in distribution to the unique solution of

(3.7)

where W i s standard Brownian motion independent of B and D.

Proof. satisfy

We begin by verifying the uniqueness of the solution of (3.7). Let f i t )

(3.8)

398 BRANCHING mocEssEs

for t < r = e-8(r) dD(s). Then

(3.9)

It follows that Z(t) = eec');?(fo e-B(s) dD(s)), where 2 is the unique solution of

(3.10)

Note that 2 is the diffusion arising in Theorem 1.3, with u2 = 1. See Theorem 1.1 of Chapter 6. By Corollary 1.9 of Chapter 3, we may assume for all t > 0,

(3.1 I ) lirn sup I B,(s) - B(s)( = 0

lim sup I D,(s) - D(s) I = 0

m-m S S I

(I-.- 191

lim f' (k - 1)2A;(ns) ds = 0 a-m 0 k > E n

Since the A; are independent of the 5 , it is enough

as.,

as.,

as.

to prove the theorem under the assumption that the A; are deterministic and satisfy (3.11). This amounts to conditioning on the A;. With this assumption we have that

(3.12) ~[z,(t)] = e4"'

and V, E so e-Bn") dU,(s) is a square integrable martingale with

(3.13)

Fix T > 0. Let T,,(I) satisfy

(3.14)

for t < r, = e-z8ncr)Z,(s) dD,(s), let Wo be a Brownian motion independent of all the other processes (we can always enlarge the sample space to obtain Wo), and define

t"(l)

e - 2~n(r)Z,(s) dD,(s) = t

(3.15)

Then W, is a square integrable martingale with (W,, W,), = t , and

3. BRANCHING FROCIESSES IN RANDOM ENVIRONMENTS 399

Since ( W,, W,), = I for all n, to show that W, * W using Theorem 1.4(b) of Chapter 7 we need only verify

(3.17)

for all f > 0. Setting 6, = suposfs I B,(t) I , we have

lim E sup I W,(s) - Wn(s-)lz = 0 n-m [ asf 1

1 I W,(S) - W,(S-))*

= e2h*a:n-2 + eZbn ( k - l)2Af')(ns)E[Z,(s)] ds 6 ' k > s .

6 ' k W e w

C (k - 1)2Af''(n~) ds. 5 e2bma,2 - 2 + e3bm

The right side goes to zero by (3.11) and the hypotheses on a,. Since Z,(r)e-B-"' is a martingale,

(3.19)

and relative compactness for {Z.} (restricted to [0, TI) follows easily from (3.16) and the relative compactness for {W#}. If a subsequence {Z,,) converges in distribution to Z, then a subsequence of

{(Wu * Po ~ X P { - 2 ~ n , ( s ) I Z n b ( ~ ) dDnb(s)))

converges in distribution to (W, ro exp { -2B(s) }Z(s ) dD(s)), and (3.16) and the continuous mapping theorem (Corollary I .9 of Chapter 3) imply

(3.20)

for t s T. The theorem now follows from the uniqueness of the solution of 0 (3.20) and the fact that T is arbitrary.

3.2 Example Let ( ( t ) be a standard Poisson process. Let A;1(f) = I , A;(t) H I + n - 'Iz( - and A: = 0 for k # 0, 2. This gives

(3.21)

400 BRANCHING PR0CESSF.S

and

(3.22) D,(i) = l ( 2 + n - ' I2( - l)t'n'k) ds.

Then (B , , D,,) 9 (B, D) where B is a standard Brownian motion and Qt) = 2r. (See Problem 3 of Chapter 7). The limit 2 then satisfies

(3.23)

4. BRANCHING MARKOV PROCESSES

We begin with an example of the t y p of process we are considering. Take the number of particles {N( t ) , t 2 0) in a collection to be a continuous-time Markov branching process; that is, each particle lives an exponentially distributed lifetime with some parameter a and at death is replaced by a random number of offspring, where the lifetimes and numbers of ofbpring are independent random variables. Note that N has generator (on an appropriate domain)

= c akp, ( f (k - 1 + 0 - f ( k ) ) I

(4.1)

where p I is the probability that a particle has I offspring. In addition, we assume each particle has a location in R' and moves as a

Brownian motion with generator +A, and the motions are taken to be indepen-

4. ERANCHING MARKOV PROCESSES 401

dent and independent of the lifetimes and numbers of offspring. We also assume that the offspring are initially given their parent's location at the time of birth. The state space for the process can be described as

(4.2) E = ((k, X I , X l r xk): k 0, I , 2, . . . I XI E R'};

that is, k is the number of particles and the x , are the locations. However, we modify this description later.

Of course it may not be immediately clear that such a process exists or that the above conditions uniquely determine the behavior of the process. Conse- quently, in order to make the above description precise, we specify the generator for the process on functions of the form f ( k , x, , x , , . . , , xk) = fl:= , d x , ) where g E 9 ( A ) and llgll < 1. If the particles were moving without branching, then the generator would be

(4.3) j = I If / ) I A , = {(h g(x,), 1 A d x , ) n B(-Yf) : 9 E W)* llgll < I *

I k

If there were branching but not motion, then the generator would be k k

A , = {( n g(x,), c a ( ~ x J ) ) - g(xj)) fl dxi)): Itell < 1) ,= I J = I , + J

(4.4)

where I&) = cI"po p f z', that is, cp is the generating function of the offspring distribution. The assumption that the motion and branching are independent suggests that the desired generator is A , + A , .

More generally we consider processes in which the particles are located in a separable, locally compact, metric space E,, move as Feller processes with generator B, die with a location dependent intensity a E C(E,,) (that is, a particle located at x at time f dies before time t + Ar with probability a(x ) At + o(At)), and in which the offspring distribution is location dependent (that is, a particle that dies at x produces I offspring with probability p,(x)). We assume that p f E C((E,) and define

(4.5) d z ) = 1 PI ZI, I z I 5 1. f

Note that for fixed z , cp(z) E C(Eo). We also assume X I Ip, E C(E,), that is, the mean number of offspring is finite and depends continuously on the location of the parent. We denote (d/az)cp(z) by cp'(z). In particular cp'( I ) = zf Ip , .

The order of the x I in the state (k, x l , x , , . . . , xk) is not really important and causes some notational difficulty. Consequently, we take for the state space, not (4.2), but the space of measures

E = { 6,, : k = 0, I , 2, . . . , xi E E o ] (4.6)

where 6, denotes the measure with mass one at x . Of course E is a subset of the space .&(E0) of finite, positive, Bore1 measures on E , . We topologize

k

I - I

402 OWcH(NC P I O a S S E S

A ( E O ) (and hence E ) with the weak topology. In other words, lim,,+m p,, = p if and only if

(4.7)

for every/€ C(Eo). The weak topology is metrizable by a natural extension of the Prohorov metric (Problem 6). Note that in E, convergence in the weak topology just means the number and locations of the particles converge.

Let C + ( E , ) = { I E C(E,): inf j > 0). Define

(4.8) (Q, P ) = /Q dcl, 8 E C(Eo), CC E E,

and note that for p -- Ifp I 6,, and g E C+(Eo),

(4.9)

Extend B to the span of the constants and the original domain, by defining El = 0, so that the martingale problem for {(S, Bf): f E 9 ( B ) n C+(E,)} is still well-posed. With reference to (4.3) and (4.4), the generator for the process will be

Let {S(r)} denote the semigroup generated by B. By Lemma 3.4 of Chapter 4, if X is a solution of the martingale problem for A, then for g satisfying the conditions in (4.10)

(4.1 I ) exp {<log W - Og. X W } - {<log S(T - slg, %I>)

is a martingale for 0 5 t s 7'. Note that

4. BRANCHING MARKOV noassEs 403

4.1 Lemma setting I X(t)I = (I, X ( t ) ) (i.e., 1 X I is the total population size),

(4.13)

Let X be a solution of the martingate problem for (A. a#). Then

E[lX(t)Il s 1/11 exp (tlla(cp'(1) - 1)11),

and

(4.14) IX ( t ) l exp {--iIla(v'(I) - 1)11} > x

By Gronwall's inequality

(4.18) ECexp { --A1 X ( t ) I I I X(r)ll I; A - 7 1 -exp { - A l ~ l ) ) e x p (tlla(v'(1)- cp(e-'))lI}.

Letting 1 -+ 0 gives (4.13). Let

(4.19)

From (4.13) i t follows that the convergence in (4.19) is in f! and hence M is a martingale and

(4.20) I W)I exp I -tlla(cp'(l) - 1)Il)

Chapter 2. 0

M(t) = lim i - ' ( l - M,(t)) = IX(t)l - (a(cp'(1) - I), X(s)) ds. 1-0 sb

is a supermartingale. Consequently (4.14) follows from Proposition 2.16 of

404 WNCHlNCROCfSSES

4.2 Theorem Let B, a, and Q be as above, and let A be given by (4.10). Then for v E B(E) the martingale problem for (A, v ) has a unique solution.

Proof. Existence is considered in Problem 7. To obtain uniqueness, we apply Theorem 4.2 of Chapter 4. Let X be a solution of the martingale problem for (A, v ) and define

(4.21) u(r. Q) = ECexp {(log gl x(W1. Note that u(r, a ) is a bounded continuous function on E = {g E C + ( E o ) : llgll < 1). For H E c(S) define

(4.22) rH(g) = lim ~-'(H(e-"g + (1 - e-")q(g)) - H(g) )

if the limit exists uniformly in g. Observe that r is dissipative, since I-H = lim,,,, E - I ( Q ~ - f)H where Q, is a contraction. We claim that u(t, .) E 9(r) and

r - 0 +

(4.23)

To see this write

(4.24) E - '(u(tl (e-"'g + (1 - e-"')cp(g))) - dt, g))

The expression inside the expectation on the right of (4.24) is dominated by

(4.25) Ila(cp(g) - dll I I 5 2114 I X(t) I 9

so by (4.13) and the dominated convergence theorem it is enough to show

converges to zero as s-+ 0 uniformly in g for each p E E. To check that this convergence holds, calculate the derivative of (4.26) and show that it is bounded. Finally, define

(4.27) W ) H ( Q ) = W ( t ) e )

4. BRANCHING MARKOV PROCESSES 405

and note that { 9 ( t ) ) is a contraction semigroup on c(E'). The fact that (4.1 I ) is a martingale gives (for T = r )

(4.28) u(t, 8) = ECexp {<log W g , X(0)))l + r4s. S(I - s ) d ds l = y(r)uo(g) + [Y(r - s)ru(s, 9) ds.

By Proposition 5.4 of Chapter I there is at most one solution of this equation, so ECexp {(log g, X ( r ) ) } ] is uniquely determined. Since the linear space generated by functions of the form exp {(log g, p ) } for g E C+(E, ) is an algebra that separates points in A ( E , ) , it follows that the distribution of X(r) is determined, and since v was arbitrary, the solution of the martingale problem for

0 (A, v ) is unique by Theorem 4.2 of Chapter 4.

We now consider a sequence of branching Markov processes X,, n = I , 2, 3, .. ., with death intensities a,, and offspring generating functions rp,, in which the particles move as Feller processes in E , with generators B,, extended as before. We define

(4.29) 2, = n - ' x , . Note that the state space for 2, is

(4.30) L

p E 4.5,): p = n - I I = I

and that 2, is a solution of the martingale problem for

(4.3 1)

106 OUNCHlNCPROCEfm

For simplicity, we assume Eo is compact (otherwise replace Eo by its one-point compact ifica t ion).

4.3 fheortm Let Eo be compact. Let B be the generator for a Feller semigroup extended as before, and let F(.): C+(Eo)-, C(Eo). Suppose

(4.34)

(4.35)

and for each k > 0,

(4.36)

If {Zm(0)} has limiting distribution v E @(crY(Eo)), then 2, n* 2 where 2 is the unique solution of the martingale problem for (A, v ) with

(4.37)

A = {exp { -<h, P)}, exp C -<h, P))<-M - WO, P>: h E a(E) n C+(Eo)} .

4.4 Remark From Taylor's formula it follows that

(4.38) Fn(h) = a,,(pL(l) - 1)h - n-'a,, (1 - u)pi(l - n- 'hu) du hZ, 1.' so typically F(h) = ah - bh', where

(4.39)

and

(4.40)

a = lim a,,(&(l) - 1) n- m

In particular, if a, = n and cp,(z) E 4 + 4zz, then F(h) = - t h 2 . Since the integral expression multiplying hz in (4.38) is decreasing in h, (4.35) and the existence of the limit in (4.36) imply there exist positive constants Ck, k 5: 1, 2, 3, such that

Proof. We apply Corollary 8.16 of Chapter 4. For h E 9 ( E ) n c+(Eo), there exist h, E a(&) n C+(Eo) such that Iim,,-- h,, = h and limn-m B,h, = Eh.

5. m o w w 407

For n sufficiently large, llh,,II -= n and h, z f inf h = E > 0. Consequently, taking g = (1 - n-'h,) in A,,

I - e- ("*#) ( - Bh - F(h), p )

Therefore, condition (f) of Corollary 8.7 in Chapter 4 is satisfied with G, = En. The compact containment condition follows from (4.14). and it remains

only to verify uniqueness for the martingale problem. Uniqueness can be obtained by the same argument used in the proof of Theorem 4.2, in this case defining

(4.44) rff(h) = lim c - ' ( H ( ( h + &F(h))VO) - f f (h) ) . 1-0 t

The estimates in (4.41) ensure that the limit

(4.45) mcexp { - ( h , x ( w 3 = lim &-'E[exp ( - ( ( h + sF(h))VO, X ( t ) ) } - exp { - ( h , X ( t ) ) } ]

8 - O t

exists uniformly in h. 0

5. PROBLEMS

1. State and prove an analogue of Theorem 1.3 for a Galton-Watson process in independent random environments. That is, let q l , q 2 , . . . be indepndent and uniform on [0, 11. Suppose the are conditionally independent given 6 = a(q,. i = I , 2, . . .) and P { t ; = /I 6) = PXq.). Define 2, as in (1.1).

108 SMNCHINC mocfisfi

Consider a sequence of such processes {ZcnJ} determined by (Pf")} and Zfn)(0), and give conditions under which Z ( " ~ [ m - ] ) / m converges in distribution.

2. Let { X , } be as in Section 2. Represent (Z':J, z(;) using multiple random time changes (see Chapter 6), and use the representation to prove the convergence of {X,).

3. Show that D = C:([O, a) x ( - a, 00)) is a core for A given by (2.8). Hint: Begin by looking for solutions of u, = Au of the form e-"(')" sin (b(t)x + cy) and e-'"'' cos (b(t)x + cy). Show that the bounded pointwise closure of A I D contains (S, A/) for /- e - O X sin (bx + cy) and f= e-"" cos (bx + cy), and the bp-closure of @(A - A I D ) contains e-'" sin (bx + cy) and e-OX cos (bx + cy), and hence all of (s([O, a) x (- 00, m)). See Chapter 1, Section 3.

k & &(s) ds < 00 as. for all c > 0. (a) Show that the solution of (3.2) exists for all time. (b) Show that the solution of (3.2) satisfies (3.1).

5. (a) In (3.23) suppose B is a Brownian motion with generator fa/'' + b r . Show that Z is a Markov process and find its generator.

(b) In (3.23) suppose B is a diffusion process with generator fa2(x)f" + m(x)f'. Show that (2, B) is a Markov process and find its gener-

ator. 6. Let A(€,) be the space of finite, positive Bore1 measures on a metric space

€ 0 . Let lcll = dE0) and define b(P1 v ) = ddtcll, v / l v l ) + 11P1- l v l l where p is the Prohorov metric. Show that 3 is a metric on A ( E o ) giving the weak topology and that ( A ( E o ) , 3) is complete and separable if (&, r) is.

7. Let B, a, and q be as in (4.10), and let e > 0. Let Be = B(f - EB)-* be the Yosida approximation of B and let Ipl = p(Eo), that is, the total number of particles. Set

4. In (3.2), assume

g E e(Eo), in fg > 0, llsil < 1 . I (a) Show that A, extends to an operator of the form of (2.1) in Chapter 4

and hence for each p E E , the martingale problem for (Aa , 6,) has a unique solution. Describe the behavior of this process.

(b) Let p E E and let X , be a solution of the martingale problem for (A, , 6,) with sample paths in Ds[O, a). Show that ( X , , 0 < E < I } is relatively compact and that any limit in distribution of a sequence

6. NOTES 409

{X,), ~ ~ 4 0 , is a solution of the martingale problem for ( A , d,,), A given by (4.10).

For X, and Y. defined by (2.4) and (2.5) show that X , and Ke""' are martingales.

8.

6. NOTES

For a general introduction to branching processes see Athreya and Ney (1972). The diffusion approximation for the Galton-Watson process was formulated by Feller (1951) and proved by Jifina (1969) and Lindvall(l972). These results have been extended to the age-dependent case by Jagers (1971). Theorem 1.4 is due to Grimvall(I974). The approach taken here is from Helland (1978). Work of Lamperti (1967a) is closely related.

Theorem 2.1 is from Kurtz (1978b) and has been extended by JotTe and Metivier (1984). Keiding (1975) formulated a diffusion approximation for a Galton-Watson process in a random environment that was made rigorous by Helland (1981). The Galton-Watson analogue of Theorem 3.1 is in Kurtz (1978b). See also Barbour (1976).

Branching Markov processes were extensively studied by Ikeda, Nagasawa, and Watanabe (1968, 1969). The measure diffusion approximation was given by Watanabe (1968) and Dawson (1975). Also see Wang (1982b). The limiting measure-valued process has been studied by Dawson (1975, 1977, 1979), Dawson and Hochberg (1979), and Wang (1982b).

IQ

Diffusion processes have been used to approximate discrete stochastic models in population genetics for over fifty years. In this chapter we describe several such models and show how the results of earlier chapters can be used to justify these approximations mathematically.

In Section 1 we give a fairly careful formulation of the so-called Wright- Fisher model, defining the necessary terminology from genetics; wc then obtain a diffusion process as a limit in distribution. Specializing to the case of two alleles in Section 2, we give three applications of this diffusion approximation, involving stationary distributions, mean absorption times, and absorption probabilities. Section 3 is concerned with more complicated genetic models, in which the gene-frequency process may be non-Markovian. Never- theless limiting diffusions are obtained as an application of Theorem 7.6 of Chapter 1. Finally, in Section 4, we consider the infinitely-many-neutral-alleles model with uniform mutation, and we characterize the stationary distribution of the limiting (measure-valued) diffusion process. We conclude with a derivation of Ewens’ sampling formula. 410

GENETIC MODELS



1. THE WRICHTflSHER MODEL 411

1. THE WRICHT-FISHER MODEL

We begin by introducing a certain amount of terminology from population genetics.

Every organism is initially, at the time of conception, just a single cell. It is this cell, called a zygote (and others formed subsequently that have the same genetic makeup), that contains all relevant genetic information about an individual and influences that of its offspring. Thus, when discussing the genetic composition of a population, it is understood that by the genetic properties of an individual member of the population one simply means the genetic properties of the zygote from which the individual developed.

Within each cell are a certain fixed number of chromosomes, threadlike objects that govern the inheritable characteristics of an organism. Arranged in linear order at certain positions, or loci, on the chromosomes, are genes, the fundamental units of heredity. At each locus there are several alternative types of genes that can occur; the various alternatives are called alleles.

We restrict our attention to diploid organisms, those for which the chromosomes occur in homologous pairs, two chromosomes being homologous if they have the same locus structure. An individual's genetic makeup with respect to a particular locus, as indicated by the unordered pair of alleles situated there (one on each chromosome), is referred to as its genotype. Thus, if there are r alleles, A , , ..., A,, at a given locus, then there are l(r + I)/2 possible genotypes, A,A, , 1 5 i s j 5 r.

We also limit our discussion to monoecious populations, those in which each individual can act as either a male or a female parent. While many populations (e.g., plants) are in fact monoecious, this is mainly a simplifying assumption. Several of the problems at the end of the chapter deal with models for dioecious populations, those in which individuals can act only as male or as female parents.

To describe the Wright-Fisher model, we first propose a related model. Let A , , ..., A, be the various alleles at a particular locus in a population of N adults. We assume, in effect, that generations are nonoverlapping. Let P,, be the (relative) frequency of A, A, genotypes just prior to reproduction, I s ; i s j s r . T h e n

is the frequency of the allele A , , 1 I; i s r. For our purposes, the reproductive process can be roughly described as

follows. Each individual has a large number of germ cells, cells of the same genotype (neglecting mutation) as that of the zygote. These germ cells split into gametes, cells containing one chromosome from each homologous pair in the original cell, thus half the usual number. We assume that the gametes are produced without fertility differences, that is, that all genotypes have equal

412 GENETIC MODELS

probabilities of transmitting gametes in this way. The gametes then fuse at random, forming the zygotes of the next generation. We suppose that the number of such zygotes is (effectively) infinite, and so the genotypic frequencies among zygotes are (2 - dI,)pIp,, where di, is the Kronecker delta. These are the so-called Hardy- Weinberg proportions.

Typically, certain individuals have a better chance than others of survival to reproductive age. Letting wI, denote the viability of A,AI individuals, that is, the relative likelihood that an A,AJ zygote will survive to maturity, we find that, after taking into account this viability selection, the genotypic frequencies become

and the allelic frequencies have the form

(1.3)

where wJr I w,, for 1 s i cj < r. The population size remains infinite. We next consider the possibility of mutations. Letting uIJ denote the prob-

ability that an A, gene mutates to an A, gene (ul, zs 0), and assuming that the two genes carried by an individual mutate independently, we find that the genotypic frequencies after mutation are given by

where

the latter denoting the probability that an A, gene in a zygote appears as A, in a gamete. The corresponding allelic frequencies have the form

1. THE WIICHTRSHER MODEL 413

= u$%l + d k l ) f ! f i l . k v l k . I

= c u t p : . k

Again, the population size remains infinite. Finally, we provide for chance fluctuations in genotypic frequencies, known

as random generic dr$, by reducing the population to its original size N through random sampling. The genotypic frequencies Pii in the next generation just prior to reproduction have the joint distribution specified by

This is simply a concise notation for the statement that ( N P I j ) i , I has a multinomial distribution with sample site N and mean vector (NPi+i*)is,. In terms of probability generating functions,

E[ n = (c lsj f $ * ( i l ) N . i s j

We summarize our description of the model in the following diagram: reproduction selection mulation rcRulallnn

adult - zygote -----+ adult --------+ adult ------, adult

N, Pi,, P C 00, (2 - dij)Pip,, pi 00, P $ , P: 00, Pc** P?* N, P;lt pf

Observe that (1.8), (l.4), (l.5), (l.2), and (1.1) define the transition function of a homogeneous Markov chain, in that the distribution of ( f ; , ) l s , is completely specified in terms of We have more to say about this chain in Section 3. For now we simply note that if the frequencies P$* are in Hardy-Weinberg form, that is, if

(1.10)

for all i s j . then

P$’ = (2 - Gi,)p:*pf*

(1 .1 I )

by (1.9). implying that

(1.12) ( p ; , . . . , p;) - (2N)- * multinomial (ZN, (p:*, . . . , pF*)).

One can check that (1.10) holds (for all (P,,),s,) in the absence of selection (i.e., w,, = 1 for all i 5 j ) and, more generally, when viabilities are multiplicative (i-e., there exist u I , . . , , u, such that wlI = uI uJ for all i 5 j), but not in general.

Nevertheless, whether or not (1.10) necessarily holds, (1.12), (1.6), (1.5), and (1.3) define the transition function of a homogeneous Markov chain, in that the distribution of ( p i , ..., is completely specified in terms of (PI, ..., p r - l). (Note that p, = 1 - z:: p l . ) This chain, which may or may not be related to the previously described chain by (l.l), is known as the Wright- Fisher model. Although its underlying biological assumptions are somewhat obscure, the Wright-Fisher model is probably the best-known discrete stochastic model in population genetics. Nevertheless, because of the complicated nature of its transition function, it is impractical to study this Markov chain directly. Instead, it is typically approximated by a diffusion process. Before indicating in the next section the usefulness of such an approach, we formulate the diffusion approximation precisely.

Put Z, = {0, 1, ...} and

(1.13) r - 1

KN = {(2N)-'a:,ct E (Z+)'-', a, 5 XV]. I = 1

Given constants p,, 2 0 (with pII = 0) and ull (= a,,) real for i, j = 1, ..., r, let {ZN(k), k = 0, 1, . . .} be a homogeneous Markov chain in KN whose transition function, starting at (PI, ..., E K N , is specified by (1.12), (La), (1.5), (1.3). and

(1.14) ulj= C(2N)-1~iJAr-1, w i j = C1 +(2W-'u,jlVh

and let TN be the associated transition operator on C(K,), that is,

(1.15) W ( P 1 . ...( Pr-1) = ECfWI, -..* p:-111. Let

(1.16)

and form the differential operator

(1.17)

where

2. ArCllCATlONS OF THE D(RuSI0N AmOXlMATION 415

Let (T( t ) } be the Feller semigroup on C(K) generated by the closure of A = ((f, G/):/E Cz(K)} (see Theorem 2.8 of Chapter 8). and let X be a diffusion process in K with generator A (i.e,, a Markov process with sample paths in CKIO, 00) corresponding to { T(t)}) .

Finally, let XN be the process with sample paths in DKIO, 00)defined by

( 1.20) X y f ) = ZN([2Nt]).

1.1 Theorem. Under the above conditions,

(1.21) lim sup sup I Tlf"y(p) - T(t)f(p)) = 0

for every SE C(K) and to 2 0. Consequently, if X N ( 0 ) * X ( 0 ) in K, then X N =+ X in DKIO, 00).

N - m Osts to P O K N

Proof. To prove (1.21). i t suflices by Theorem 6.5 of Chapter I to show that

(1.22) lim sup 12N(TN - I)s(p) - Gf(p)( = 0

for allfE C'(K). By direct calculation,

N - m p c K n

(1.23)

(1.24)

(1.25)

2NEcP; - Prl = Mp) + O ( N - l ) ,

2N cov (p;, p;, = a&) + O ( N - I),

2NEC@; - pi)*] = O ( N - ' ) ,

and hence

( 1.26)

(1.27)

as N-, 00, uniformly in p E K N , for i , / = I, ..., r - I and E > 0. We leave to the reader the proof that (1.23), (1.26), and (1.27) imply (1.22) (Problem I).

The second assertion of the theorem is a consequence of (1.21) and Corol- lary 8.9 of Chapter 4. 0

2NEC(/.J; - PAP; - PJI = a , h ) + O(N -

2NP( IP; - PI1 > E } = O(N-9,

2. APPLICATIONS OF THE DIFFUSION APPROXIMATION

In this section we describe three applications of Theorem 1 . 1 . We obtain diffusion approximations of stationary distributions, mean absorption times, and absorption probabilities of the one-locus, two-allele Wright-Fisher model. Moreover, we justify these approximations mathematically by proving appropriate limit theorems.

416 CENmC MoDEui

Let {ZN(k) , k = 0, 1, . . .} be a homogeneous Markov chain in

K , = {L* i = 0, 1, ..., 2N 2N'

whose transition function, starting at p E KN, is specified by (1.12), (1.6), (1.5), (l.3), and (1.14) in the special case r = 2. Concerning the parameters pr2, pzl, uI1, u , ~ , and oz2 in (l.14), we assume that oI2 = 0 and relabel the remaining parameters as p,, p 2 , ol, and o2 to reduce the number of subscripts. (Since all viabilities can be multiplied by a constant without affecting (1,3), it involves no real loss of generality to take w12 = 1, i.e., olZ = O.)Then ZN satisfies

(2.2) P p ( k + 1) = 2- 2N p ( k ) = = (2jN>@**Ml - p*+)"-',

where

(2.3) p+* (1 - u&+ + Ua(1 - p"),

(2.4) P+ = W , P 2 + P(1 - P)

w,p' + 2p(l - p) + w,(l - p)l'

and

(2.5) ui = [ (2N) - 'p i ]A4 , wi = [I + (2N)-'a,] V i , i = 1, 2.

Recalling the other notation that is needed, TN is the transition operator on C(K,) defined by (I . 1 S),

(2.6) K = LO, 13, and XN is the process with sample paths in D,,[O, 00) defined by (1.20). Finally, { T(r)} is the strongly continuous semigroup on C(K) generated by the closure of A = {(I; G f ) : f ~ C'(K)}, where

(2.8)

and

(2.9)

and X is a diffusion process in K with generator A. Clearly, the conclusions of Theorem 1.1 are valid in this special case. As a first application, we consider the problem of approximating stationary

distributions. Note that, if bl, pa > 0. then 0 < p++ < 1 for all p E K,, so ZN is an irreducible, finite Markov chain. Hence it has a unique stationary distribution v, E 9 ( K N ) . (Of course, we may also regard vN as an element of 9(K). ) Because vN cannot be effectively evaluated, we approximate it by the stationary distribution of X .

a(P) = P(1 - PI,

b) = -PIP + Pz(1 - P) + P(1 - P ) P , P - @2(1 -PI].

2 lVPLlCATIONS OF THE DIRUSION APPROXIMATION 417

2.1 Lemma Let pl, pz > 0. Then X has one and only one stationary distribution v E B(K). Moreover, v is absolutely continuous with respect to Lebesgue measure on K, and its density h, is the unique Cz(O, I ) solution of the equation

(2.10) t(ah0)” - (bho)’ = 0

with lA h,(p) d p = 1. Consequently, there is a constant /3 > 0 such that

(2.1 I ) h,(p) = /3pZ”*-’( l - p ) * ” ’ - ’ exp { o l p 2 + a,(l - p12}

for 0 c p c 1.

Proof. We first prove existence. Define h, by (2.11), where D is such that ho(p) dp = I , and define v E 9 ( K ) by v(dp) = h,(p) d p . Since (ah,)(O+) =

(ah,MI -) = 0 and (2.10) holds, integration by parts yields

(2.12) pv = 0, / € CZ(K).

It follows that

(2.13)

Thus, v is a stationary distribution for X. (See Chapter 4, Section 9.)

define

@ d v = 0 for all/€ 9(A), and hence

& T ( t ) / d v = I / d v . / E CW), t 2 0.

Turning to uniqueness, let v E 9 ( K ) be a stationary distribution for X, and

(2.14)

Since (2.13) holds, so does (2.12). In particular, dl) = 0 (take/@) = p) , so

(2.15)

for everyf E C’(K). Therefore,

418 GENETIC MODELS

as Bore1 measures on K. Since u > 0 on (0, I), we, have v(dp) << d p on (0, l), so by (2.14), c is continuous on LO, I). By (2.16), v(dp)ldp has a continuous version ho on (0, l), and +aho = c there. Thus, by (2.14),

(2.17)

for 0 < p < 1. It follows that ho E C'(0, 1) and (2.10) holds, so since h,, is Lebesgue integrable, it is easily verified that ho has the form (2.11) for some constant 8. To verify that is such that j k ho(p) d p = 1, and to complete the proof that v is uniquely determined, it suffices to show that v({O}) = u({ I}) = 0. By (2.1 l), (aho)(O+) = 0, so by (2.17), c(0) = 0. Since b(0) = p2 > 0, we have ~((0)) = 0 by (2.14). Similarly, c(l -) = 0, so v({l}) = 0, completing the proof.

0

We now show that v N , the stationary distribution of ZN, can be approximated by v, the stationary distribution of X.

2.2 Theorem. Let pl , pa > 0. Then vN =+ v on K.

Proof We could essentially quote Theorem 9.10 of Chapter 4, but instead we give a self-contained proof. By Prohorov's theorem, (v,,} is relatively compact in P(K), so every subsequence of ( v N ) has a further subsequence { v N . } that converges weakly to some limit i, E P(K). Consequently, for all f E C(K) and t 2 0,

l T ( t ) f d0 = lim T ( r ) f d v w (2.18) 1.. N'- w

= lim L N , T p N " y d v w

= lim l , / t i v , .

= If d5,

"-roo

"+m

K

so 3 is a stationary distribution for X. (This gives, incidentally, an alternative proof of the existence of a stationary distribution for X.) By Lemma 2.1, 0 = v.

0 Hence the original sequence converges weakly to v.

2.3 Remark If ot = uz = 0, then the stationary distribution of Theorem 2.2 belongs to the beta family. In particular, its mean is pz/(p, + pa), which also happens to be the stable equilibrium of the corresponding deterministic model

0 b = -PIP + P2U -PI.

2. APPLICATIONS OF THE DIFFUSION AWROXIMATION 419

If p1 = 0 (respectively, if p2 = 0), then 1 (respectively, 0) is an absorbing state for the Markov chain Z N , so interest centers on the time until absorption and (if p1 = p2 = 0) the probability of ultimate absorption at 1. Of the three cases, p , = p2 = 0, p l = 0 < pz , and p , > 0 = p 2 , it will sufice (by symmetry) to treat the first two. Let

(2.19) F = (0, I ) if pl = P Z =o, F = { l } if p 1 = 0 , p 2 > 0 .

Then F is the set of absorbing states of ZN (and hence X'), and i t is easily seen (by uniqueness, e.g.) that F is also the set of absorbing states of the diffusion X.

((x) = inf ( t z 0: x(t) E F or x(r-) E F }

Define C: DJO, a)-+ LO, a] by

(2.20)

where inf 0 = a. Then C is Bore1 measurable, so we can define

(2.21) T N = ( (XN), T = C(X).

In order to study the mean absorption time E[TN] and (if p, = p2 = 0) the absorption probability P{XN(zN) = I } , we regard Err] and P { X ( r ) = I } as approximations, the latter two quantities being quite easy to evaluate. The following theorem is used to provide a justification for these approximations.

2.4 Theorem in DKIO, a)) x 10, a)], and the sequence ( T N ) is uniformly integrable.

Let p , = 0, p2 2 0. If XN(0) - X ( 0 ) in K , then ( X N , r N ) * (X, T)

We postpone the proof to the end of this section.

2.5 Remark It follows from Theorem 2.4 that, for each N 2 1, T~ and T have finite expectations, hence they are as. finite, and therefore XN(rN) and X(T) are defined as. and equal to 0 or 1 a.s. These facts are needed in the corollaries that follow. It is also worth noting that the first assertion of Theorem 2.4 is nor a consequence of Corollary 1.9 of Chapter 3 because the function x w ( x , ((x)) on DKCO, 00) is discontinuous at every x E DJO, 00) for which ((x) < 00, hence

0 discontinuous as. with respect to the distribution of X .

2.6 Corollary Let p , = 0, p2 2 0. If XN(0) - X(0) . then

(2.22)

Proof. By Theorem 2.4, T ~ T * T , so (2.22) follows from the uniform integrability of {fNJ. 0

420 CENETKMOOBLS

2.7 Corollary Let pl = pa = 0. If X"(0) sg X(O), then

(2.23)

Proof. Define €: DKIO, ao) x [0, m]-+ K by t ( x , r ) = x(r ) for 0 5 t < WJ and {(x, ao) = 4, say. Then € is continuous at .each point of CJO, ao) x [0, oo), hence continuous a.s. with respect to the distribution of ( X , T), By Theorem 2.4 of this chapter and Corollary 1.9 of Chapter 3, XN(rN) rg X(T), so (2.23) follows from Remark 2.5. 0

To evaluate the right sides of equations (2.22) and (2.23), we introduce the notation Pp{ .} and E,[ -1, where the subscript p denotes the starting point of the process involved in the probability or expectation.

2.8 Proposition Suppose first that p1 = p, = 0. Let /b be the unique C 2 ( K ) solution of the differential equation Gfo = 0 with boundary conditionsfo(0) = O,fo(l) = 1. Then P p ( X ( t ) = 1) -job) for all p 6 K. Consequently,

(2.24)

where J(q) = a,q2 + a2(l - 4)'.

solution of the differential equation Gg, = - 1 with boundary conditions

(2.25)

Then E,[r] = go(p) for all p E K. Consequently, if p, = 0, then

Now suppose that pl = 0, p2 2 0. Let go be the unique C(K) n C2(K - F)

go(0) = go(1) = 0, if = 0,

gb(0 +) finite, go( 1) = 0, if p2 > 0.

and, if p2 > 0, then

Proof. For each /E C2(K), p 6 K, and I 2 0, the optional sampling theorem implies that

(2.28)

1. APPLICATIONS OF THE DIFFUSION APPROXIMATION 421

Replacingfbyj, and letting I-+ ao, we get P,{X(r) = I } =fo(p) for all p E K. Here we are using Remark 2.5.

Given h E C(K), let g be the unique C ( K ) n C2(K - F ) solution of the differential equation Gg = - / I with boundary conditions analogous to (2.25). Then g = Bh, where

if p2 = 0, and

Bh(p) = q ' - 2 # 2 e - " q ' 2r2NZ-I ( I - r ) - I e'"'h(r) dr dq (2.30) S , ' l if pz > 0. Consequently, Eh E C1(K) if h E C 1 ( K ) and h = 0 on F. Thus, we choose (h,} c C'(K) with h, z 0 and h, P I - zF. Replacing/ by g , = Eh, in (2.28), and noting that bp-lim g. = go, we obtain

(2.31) EpCgo(X(T A t))I = go@) - EpC? A 13

for all p E K and r L 0. Letting I .-+ a0 gives E, [T] = go@) by (2.25). We leave it to the reader to verify (2.24), (2.26). and (2.27). 0

2.9 Remark As simple special cases of Proposition 2.8, one can check that

a = 0,

(2.32)

ifp! = p1 = Oand oI = -u2 = a,and

(2 .33) E,bI = - 2CP log P + (1 - P ) log ( 1 - PI1 if p , = p 2 = u, = u2 = 0. To get some idea of the effect that selection can have, observe that, when p = ), the right side of (2 .32) becomes 1/(1 + e-") , and in view of (2.5), I ol may differ significantly from zero. When interpreting (2.33) (or, more generally, (2 .26) or (2.27)), one must keep in mind that, because

0 of (1.20). time is measured in units of 2N generations.

In order to prove Theorem 2.4, we need the following lemma.

2.10 Lemma Let p, = 0, p2 2 0, and define the function go as in Proposition 2.8. Then there exist positive integers, K and N o , depending only on p 2 , 0,.

and u 2 , such that

422 CWTlC MODLLS

Proof. Define the operator GN on C(KN) by

(2.35) G N . f ( P ) 2N{Epcf l zN(1)) l -s(p))*

For 0 s E < i, let

(2.36) (6, I - E ) if p2 = 0 [O, 1 - E ) if p2 > 0,

and put VN(&) = KN A V(E). (Note that V(0) = K - F.) The first step in the proof is to show that

(2.37) Iim & Sup GN&) < - 1. m-ao N - S A p 6 V N ( m / l N )

A fourth-order Taylor expansion yields

I + - 31 E p [ ( z N ( U - pI4 l(1 - tI3g$"(p + t(zN(l) - PI) d t ] )

for all p 6 VAO) and N 2 1. (We note that the integral under the fourth expectation exists, as does the expectation itself.) Expanding each of the moments about pa*, which we temporarily denote by y, we obtain

2N 3yz(1 - 7)' + y(1 - 7)(1 - 67 + 67') + "' ' 6 [ (2N)' ( W

x L(I - 0 3 SUP I ~ P ( P + t(4 - PI) I dt OS#S 1

for all p E VN(0) and N 2 1, where I O N . 5 1. Now one can easily check that

(2.40)

and

2N(p** - p ) = bO(1 + O ( N - 9 ) + O(pz(1 - p)"-')

(2.41) p**(1 - p**) = a(pw1 + O ( N - ' ) ) + O(pZ(1 - p) 'N- ' )

2. APPLICATIONS OF THE DIFFUSION APPROXIMATION 423

as N -+ Q), uniformly in p E K,. Also, by direct calculation, there exist constants MI, .. ., M4, depending only on p z , a,,and a2,such that

(2.42)

for all p E V(0) and k = 2, 3,4. Finally, we note that

since the minimum occurs at 9 = 0 or q = I , and therefore

for all p E V(0). By (2.39H2.42) and (2.44), we have

as N -+ m, uniformly in p E VN(0), which implies (2.37) since Gg, = - 1. Next, we show that

(2.46) - limG,,go(l -g)<O, m = 1,2 ,.... N - m

Fix m 2 I , and let pn = (1 - m/2N) VO. Since gi is bounded on (0, 4) if p2 > 0, there exists a constant M,, depending only on p z , qlr and n 2 , such that

-2 5- + Mo log

1-P

424 GENETIC MODELS

for all p E V(0). Copsequently, a second-order Taylor expansion yields

- 4NEpN[(ZN(1) -pN)' l(1 - tN1 - Y)- l dt I for each N 2 1, where Y = pN + r(ZN(l) - pN). Using (2.42) and (2.43), the first two terms on the right side of (2.48) are O(N-' log N) as N 4 00, so (2.46) is equivalent to

Denoting p+* by pa* when p = pN, the expectation in (2.49) can be expressed as

and since 1 - p;t+ = m/2N + O(N-'), an application of Fatou's lemma shows that the left side of (2.49) is at least as large as

(2.51)

which of course is positive; here we have used the familiar Poisson approximation of the binomial distribution. This proves (2.46), and, by symmetry,

(2.52) - lim GNgo($) < 0, m = 1, 2, ...,

N-tm

if p2 = 0. Combining (2.37), (2.46), and (if p2 P 0) (2.52), we conclude that there exist K and No such that

1 (2.53) GNgO(P) 5 - i s p € VdO), N 2 No.

2. APPLICATIONS OF THL DlRUSlON AFPROXlMATlON 425

Finally, to complete the proof of the lemma, we note that

is a martingale, so by the optional sampling theorem and (2.53),

for all p e VN(0), t = 0, 1/2N, 2/2N, . .. , and N ;r No, and this implies (2.34). 0

Proof of Theorem 2.4. For 0 _< E < 4, define r : DK[O, a)--+ [O, a3 by

(2.56) r ( x ) = inf { t 2 0: x(t ) $ V(E) or x ( t - ) # V(E) )

where Y(E) is given by (2.36). (Note that lo = C; see (2.20).) Then C(x)-+ ((x) and ~ - - r 0 for every x E CKIO, co), hence as . with respect to the distribution of X . In addition, we leave it to the reader to show that c' is continuous as. with respect to the distribution of X for 0 < E < i(Problem 3).

We apply the result of Problem 5 of Chapter 3 with S = D,[O, oo), S' = D,[O, 00) x [O, 003, h(x) = (x, Ax)), h,(x) = (x, CYx)), where 0 < c4 < 4 and E ~ - + 0 as k-, CQ. To conclude that h(XN) * h(X) , that is, ( X N , T N ) 3 (X, T), we need only show that

(2.57)

lor every 6 > 0, where p ( f , t ' ) = I tan - I f - tan ' t' I . By the strong Markov property, the inequality I tan t - tan - ( I + s) I 5 s for s, t ;r 0, and Lemma 2.10, we have

(2.58) %(f"(XN), C(X" > 6)

s E L X I , ; c PX*(r:) {TN > d)] 5 6 - ' sup Ep[rN]

p c K N n V(KF

s S - l K SUP go(p) p e K n V(cF

for all S > 0, N 2 N , , and 0 < E < 4, where T L = c ( X N ) . Since go = 0 on F, (2.57) follows from (2.58).

Finally, we claim that the uniform integrability of ( T , } is also a consequence of Lemma 2.10. Let g o , K , and No be as in that lemma. Then

(2.59) sup P P ( t N > t } s t - sup EP[r,] I I- sup go(p) p c K N P C K N v * K

426 CPNTnC MODELS

for all N 2 N o and r > 0, so there exist ro > 0 and q < 1 such that

sup sup P,,{rN > lo) < q. NLI p e K n

Letting Ef: = { T ~ > m/2N}, we conclude from the strong Markov property that, if n ;t [2Nt,] , then

(2.61)

for each m 2 0, p E KN, and N 2 1. Consequently, for arbitrary 1 > I,

P A G + “1 = &CX$ pxN(n,2N)(~nN)l s rlPp(E3

(k + I)n

(2.62) E p [ ( r N ) ‘ l = LEO / = k n + l ( & y p p { r N = &}

m

s C (k + 1)’d P,(E,N,) k - 0

k - 0 ,

for all p B KN and N 2 1, where n = [2Nro]. Since the bound in (2.62) is uniform in p and N, the uniform integrability of {tN} follows, and the proof is complete. 0

3. GLNOTYPIC-FREQUENCY MODELS

There are several one-locus genetic models in which the successive values (from generation to generation, typically) of the vector (P,,),, of genotypic frequencies form a Markov chain, but the successive values of the vector (p,, . . . , p,- of allelic frequencies do not; nevertheless, the genotypic frequencies rapidly converge to Hardy-Weinberg proportions, while, at a slower rate, the allelic frequencies converge to a diffusion process. Thus, in this section, we formulate a limit theorem for diffusion approximations of Markov chains with two “time scales,” and we apply it to two models. Further applications are mentioned in the problems.

Let K and H be compact, convex subsets of R“ and W, respectively, having nonempty interiors, and assume that 0 E H. We begin with two lemmas involving first-order differential and difference equations, in which the zero solution is globally asymptotically stable.

3.1 Lemma Let c : K x R”-r R” be of class C2 and such that the solution Y(t , x, y) of the differential equation

(3.1) d dr - w, x, Y) = 4% Y(t, x, YlX Y(0, 3, Y) = Y,

3. CENOTVPIC-FREQULNCV MODf3.S 427

exists for all ( r , x, y ) E [O, GO) x K x H and satisfies lim sup I Y(r, x , y) l = 0. 1-m ( x . p ) e K n H

Then there exists a compact set E, with K x H c E c K x R", such that (x , y ) E E implies (x , Y(t , x , y ) ) E E for all I 2 0, and the formula

(3.3) S(t)h(x, y ) = h(x, W, x , Y ) )

defines a strongly continuous semigroup {S(r ) ) on C(E) (with sup norm). The generator B of {S(t)} has Cz(E) = { . f I E : / ~ Cz(R" x R")} as a core, and

n a I - I aYl

(3.4) Bh(x, y ) = cdx, y) - h(x, y ) on K x H, h E Cz(&).

Finally, lim sup IS(t)h(x, y ) - h(x, 0)l = 0, h E C(E). I-, (I. y ) 6 €

(3.5)

?roof. Let E = {(x, Y(t , x, y ) ) : ( t , x, y ) E [O, 00) x K x H } . By (3.2), E is bounded, and E is easily seen to be closed. If (x, y ) E E, then y = Y(s, x, yo) for some s 2 0 and yo E H. Hence ( x , Y(t , x , y ) ) = ( x , Y(t + s, x, yo)) E E for all I 2 0, and (3.6) lim sup I Y(r, x , y)l < lirn sup sup 1 Y(t + s, x , y o ) ( = 0

by (3.2). It is straightforward to check that ( S ( r ) } is a strongly continuous semigroup on C(E). By the mean value theorem, C2(E) c O(B) and (3.4) holds. Since S(r): Cz(E)-, C2(E) for all t 2 0, C2(E) is a core for B. Finally, (3.5) is a

1-mJ ( x . p ) e E ( - 0 0 .LO ( x . y o ) e R N H

consequence of (3.6). 0

3.2 Remark If c(x, y ) = cp(x)y for all ( x , y ) E K x W", where cp : K -+ BB" 8 W" is of class C2, and if for each x E K all eigenvalues of p ( x ) have negative real parts, then c satisfies the hypotheses of Lemma 3.1. In this case, Y(r, x, y ) E

0

3.3 lemma Given 8 , > 0, let c : K x 08"- W" be continuous, such that the solution Y(k, x, y ) of the difference equation

(3.7) a,'{ Y(k + 1, x, Y ) - Y(k, x , Y ) } = c(x, Y(k, x, Y h

(3.8)

Y(0, x , y ) = y,

which exists for all (k , x, y ) E Z + x K x H, satisfies lim sup I Y(k, x, y)l = 0.

Then there exists a compact set E, with K x H c E c K x R", such that (x, y) E E implies (x , Y(k, x, y) ) E E for k = 0, 1,. . . , and the formula

(3.9)

k - m ( x , p ) e X M H

S(t)h(x, Y) = ECh(X. Y ( W . XI Y))l,

428 CENmC MOOflS

where V is a Poisson process with parameter 6;'. defines a strongly continuous semigroup (S(r)} on C(E). The generator B of {S(r)} is the bounded linear operator

(3.10) B = 6,'(Q - I ) ,

where Q is defined on C(E) by Qh(x, y ) = h(x, y + 6, c(x, y)). Finally,

(3.1 1) lim sup I Qkh(x, y ) - h(x, 0) I = 0, h E C(&). k + Q k y).&

Proof. Let E = {(x, Y(k, x, y) ) : (k , x, y ) E Z, x K x H}. The details of the 0 proof are left to the reader.

3.4 Remark If dx, y) = cp(x)y for all (x, y) E K x W", where cp: K 4 R" QD W" is continuous, and if for each x E K all eigenvalues of q(x) belong to {I E @: I C + S,' I < d:'}, then c satisfies the hypotheses of Lemma 3.3. In

0 this case, Y(k, x, y ) = ( I + S, (p(x))'y.

The preceding lemmas allow us to state the following theorem. Recall the assumptions on K and H in the second paragraph of this section.

3.5 Theorem For N = 1, 2, ..., let {ZN(k) , k = 0, 1, ...} be a Markov chain in a metric space EN with a transition function pN(z, r), and denote f(z')p&, dz') by E,[f(ZN(l))]. Suppose both a,,: EN-' K and VN: EN+ N

are Bore1 measurable, define X N ( k ) = U+,@'"k)) and YN(k) = V&ZN(k)) for each k 2 0, and let E~ > 0 and > 0. Assume that limN-,,, SN = 6, E [0, 00) and l im, , ,~~/S~ = 0. Let each of the functions a: K x W"- R'"@R"', b: K x R"-, R", and c : K x R"-, W" be continuous, and suppose that, for i, j = I, ..., mandl= 1, ..., n, (3.12)

(3.13) 'E,C(X,N(l) .- xiHX,N(l) - x,)I = Y) + dl), (3.14)

(3.15)

(3.16)

&i 'E .CXW) - xrl = bXx, Y) + 4l),

E N '~,C(XW) - X S I = 41). ~,'E*CY,N(1) - Yrl = cxx, Y) + 411, s,'E,C(Y,N(1) - E,CYX111)'1 = 411,

as N - 00, uniformly in z E E N , where x = QN(d and y = VN(z). Let

(3.17)

and assume that the closure of {(f, G f ) : f ~ C2(K)) is single-valued and generates a Feller semigroup { U(t ) } on C ( K ) corresponding to a diffusion process X in K. Suppose further that c satisfies the hypotheses of Lemma 3.1 if 6, = 0 and of Lemma 3.3 if 6, > 0. Then the following conclusions hold :

3. CENOTYPIC-FREQUENCV MODELS 429

(a)

(b)

If X N ( 0 ) * X(0) in K, then XN([ */EN]) * X in DJO, 03).

If {t,,,} c [0, 00) satisfies limN4, I, = a, then Y N ( [ t N / 6 , ] ) - 0 in H.

3.6 Remark (a) Observe that (3.12H3.14) are analogous to (1.23), (1.26), and (1.25), except that the right sides of (3.12) and (3.13) depend on y. But because of (3.15), (3.16), and the conditions on c, it is clear (at least intuitively) that conclusion (b) holds, and hence that Y’([t/&N]) * 0 for each t > 0. Thus, in the “slow” time scale (ie., t / eN) , the Y N process is “approximately ” zero, and therefore the limiting generator has the form (3.17).

(b) We note that (3.14) implies

(3.18) E n ’ P * { l X W ) - x , I > ~ ) =o( l ) , Y > O ,

tion suffices in the proof.

for i = 1, ..., m. (Here and below, we omit the phrase, ‘‘as N-* 00, uniformly in t E EN, where x = @,(z) and y = ‘f”(~).”) In fact the latter condi-

0

Proof. Let E be as in Lemma 3.1 if 8 , = 0 and as in Lemma 3.3 if 6, > 0, and apply Theorem 7.6(b) of Chapter 1 with LN = B(E,,,) (with sup norm), L = C(E), and nN: L-, L,,, defined by n N S ( t ) = j ( x , y). where x = mN(z) and y = V&). Define the linear operator A on L by

and let B be the generator of the semigroup (S(t)} on L defined in Lemma 3.1 if 6, = 0 and in Lemma 3.3 if 6, > 0. Define P on L by Ph(x, y ) = h(x, 0), and let D = Ed(A) n 9 ( P ) and D’ = C’(E). By the lemmas, D’ is a core for B. and (7.15) of Chapter 1 holds if 6, = 0, while (7.16) of that chapter holds (where Q is as in Lemma 3.3) if 6, > 0. Let AN = EN ‘(TN - I), where T, i s defined on LN by T’,f(z) = E,[f(P(zN( I))], and k t d lN = 6~ /+ , .

Given f E D,

(3.20) A N ~ N J ( ~ = &,i’E,Cf(XN(I), Y) -AX, Y)I M

= c E N ‘E,CX,N( 1) - W J X , Y) I = I

430 GENETICMOOUS

where the first equality uses the fact thatfE d?(P), the second uses the convexity of K, and the third depends on (3.12), (3.13), and (3.18). (To show that the remainder term in the Taylor expansion is o(l), integrate separately over I X N ( l ) - X I I; y and I XN(l) - X I > y, using the Schwarz inequality, (3.13), and (3.18).) This implies (7.17) of Chapter 1.

Given h E D',

where

I

(1 - u)g(x + u(XN(1) - x), EJYN(l)] + U(YN(1) - Er[YN(l)])) du. -1 (Here the convexity of H and of K x H is used.) But the right side of (3.21) is o(1) by the Schwarz inequality, (3.12), (3.13), and (3.16). Consequently,

(3.23) a,i lANnN&) = &'{h(x, EzW"(l)l) - h(x, Y ) ) + 41) = W x , Y ) + 41)

by (3.15) and either (3.4) or (3.10). This implies (7.18) of Chapter 1. Finally, define p : K- E by p(x) = (x, O), and observe that

G ( f 0 p) = (PAf) o p for all PE D. Since the closure of {(A G f ) : f e C'(K)J is single-valued and generates a Feller semigroup { tr( t )} on C(K), the Feller semigroup { T(r)} on b =i w(P) satisfying

is generated by the closure of P A I D . Theorem 7.qb) of Chapter 1, together with Corollary 8.9 of Chapter 4, yields conclusion (a) of the theorem, and

U Corollary 7.7 of Chapter 1 (with h(x, y ) = I y I) yields conclusion (b).

3. GENOTYPIC-FREQWNCY M O O n S 431

3.7 Remark (a) Since limN+m eN = 0, (3.12) implies that (3.13) is equivalent to

for i, j = 1, . . ., m and that (3.14) is equivalent to

(3.26) for i = 1, ..., m. It is often more convenient to verify (3.25) and (3.26). We note also that, if IimNdm dN = 0, then (3.15) implies that (3.16) is equivalent to

& i ' E , C ( X W ) - E,CXIY(1)l)41 = 41)

(3.27) &LEzC(YIN(l) - Yr)21 = 41) for I = I , . . . , n.

(b) It is sometimes possible to avoid explicit calculation of (3.16) by using the following inequalities. Let C and g be real random variables with means [and ij such that I t I 5 M a.s. and I g I < M a.s. Then

(3.28) var (t + q) s 2(var 4 + var g)

and

(3.29) var (CV) s E C ( h - m21 s 2E[(C - .52g21 + 2PE[(q - ip] s 2M2(var t: + var g). 0

In the remainder of this section we consider two genetic models in detail, showing that Theorem 3.5 is applicable to both of them. Although the models differ substantially, they have several features in common, and it may be worthwhile pointing these out explicitly beforehand.

Adopting the convention that coordinates of elements of R"'+11/2 are to be indexed by { (C j ) : 1 s i $ j 5 r ) , the state space E N of the underlying Markov chain ZN in both cases is the space of genotypic frequencies

(3.30)

In applying Theorem 3.5, the transformations QN: E - + R'-' and Y N : EN .-., R.(v + I )/z are given by

(3.31) @N((P,,),.,) =(PI. . * . * P r - 1 )

and

(3.32) V N ( ( P , , ~ s I) = (QI,)I 5 j 9

where p , is the allelic frequency (1.1) and Q,, is the Hardy-Weinberg deviation

(3.33) Qr, = f', - (2 - S,,)P, PI.

432 GENETIC MOOLLS

Observe that QAE,.,) c K, where K is defined by (1.16). As we see, in both of our examples, the functions a: K x W+'J'2 -+ R'-' @ W-' and b: K x Wr(r+1)12+ R'-' are such that a,,(p, 0) and b,@, 0) are given by the right sides of (1.18) and (1.19). Consequently, the condition on G in Theorem 3.5 is satisfied. In addition, the function c: K x W".+1u24 is seen to trivially satisfy the conditions of either Remark 3.2 or Remark 3.4. (Hence H can be taken to be an arbitrary compact, convex set containing

Thus, to apply Theorem 3.5, it suffices in each case to specify the transition function, starting at (PiJi,, E E N , of the Markov chain Z", and to verify the five moment conditions (3.12H3.16) for appropriately chosen sequences (8")

and {&}. Before proceeding, we introduce a useful computationaj device, which

U N Z I Vd4v)J

already appeared in (1.7) without explicit mention. Given (d,,),,, E Rr(r+1)'2 9 we

(3.34)

We apply this symmetrization to F,,, f,f, f$*, P;,, Ql,, Q;,, and so on. The point is that (1.1) can be expressed more concisely as pi = & p,,. For later reference, we isolate the following simple identity. With (d,,),,, as above,

define

4, = 91 + 6,) d ,A , . l v l , i , i = 1, ..., r.

(3.35) (611 sk, v bit 6,kNI + d,,) Jik k. I

= cdi, slrl + 611 d,k(l - 61, 6kl)l(1 + sJI> Z k k . I

= b,, dik + di,, i,i 1, s..) r. k

3.8 Example We consider first the multinomial-sampling model described in Section 1. The transition function of ZN, starting at (Pu)r,, E EN, is specified by (1.8), (1.4), (1.5), (1.2), ( l . l ) , and (1.14).

Since ECP;,] = f$*, we have (3.36) 2 N E [ p ; - pi] = 2 N ( p f + - pi) = b h ) + O(N-'),

where b: K+ R'-' is given by (1.19). (Throughout, all 0 and o terms are uniform in the genotypic frequencies.) The relation cov (P;k, Pi,) = N - ' P$*(S, 6 k 1 - fj*) implies (3.37) COV (h, Q = N-'[f(6,,&lV$S,JJO( + 8,JPi* - &*Q*I, and therefore, by (3.35), (3.38) 2N cov ( p i , p i ) = 1 2N cov (Rk, el)

k . 1

= s,,p:* + P$* - 2 p y p y

= p?*(S,, - p y , + P$* - p f * p f + .

3. GLNOTYPlC-FREQUENCV MODELS 433

This shows, incidentally, that (1.10) is not only suficient for (1.12) but necessary as well. Now observe that pf' = pi + O(N I ) by (3.36) and

1 (3.39) p:' = 5 (6kt 6,j + bltK2 - sk, )pk PI + O(N -

ks1

=pip]+ O(N- ' ) ,

so

(3.40) 2 N cov (p;i P;) = ~ i ( d , j - P j ) + O ( N - ')*

Next, we note that

(3.41) Q Q ; j - Qijl = P$" - (2 - Sij)[cov (pi P;) + P:+P,+"~ - Q i j

= - Q i j + O ( N - ' )

by (3.39) and (3.40). Finally,

(3.42) ~ N E C ( ~ ; - EC~;I)*I 2 ~ ~ 3 i E ~ P ; , - ECP;,I)~I I = I

= O ( N - 1 )

since Pij - N-' binomial (N, ECP;,]) for each i S j , and the fourth central moment of N-' binomial (N, p) is O(N-*) , uniformly in p. Also,

(3.43) var (Qi,) 4 2 var ( P -t 2(2 - 6,,)' var (pi p i )

s O(N I ) + 4(2 - d,,)'(var (pi) + var (pj ) )

= q N - 1 )

by (3.28) and (3.29). This completes the verification of conditions (3.l2H3.16) of Theorem 3.5 (see Remark 3.7(a)) with cN = (2N)- ' and 6, = 1.

We note that the limits as N - + 00 of the right sides of (3.36) and (3.40) depend only on pl, ..., P , - ~ . (For this reason, Theorem 3.5 could easily be

0 avoided here.) However, this is not typical, as other examples suggest.

3.9 Example The next genetic model we consider is a generalization of a model due to Moran. Its key feature is that, in contrast to the multinomial- sampling model of Example 3.8, generations are overlapping. A single step of the Markov chain here corresponds to the death of an individual and its replacement by the birth of another.

Suppose the genotype Ai A j produces gametes with fertility w]:' and has mortality rate wi:). If (P, , )rs , E E N is the initial vector of genotypic frequencies, then the probability that an A, A, individual dies is

(3.44)

434 CEN€llCMOMLS

The frequency of A, in gametes before mutation reads

(3.45)

where wf:) I will) for 1 s i < j s r. With mutation rates uu (where u,, = 0), this becomes

(3.46) P:* = (1 - ; Utj)P? + c u,, Pl' I

after mutation, so the probability that an A, A, individual is born has the form (3.47)

Consequently, the joint distribution of genotypic frequencies Pi, after a birth-death event is specified as follows. For each (k, I) # (m, n) (with 1 5 k I; I < r, 1 s m S n s r),

(3.48) P;, = P,, - N-' if (i, j ) = (k, r ) 1 Pi, otherwise

Pi, = (2 - 6&?*Pl'*.

P,, + N-' if (i , j ) = (m, n)

4. INFIMTELV-MANV-AUELE MODELS 435

where o,, = 01,” - 01:’ is the difference between the scaled fertility and mortality, and

(3.53) N2E[(P; - p i b ; - PI)] = N 2 - P i k M P ; , - Pji)] k . I

= [: - (?,k 8 j l + ? I f 8 i k ) k . I

+ fcai, 6 k 1 v 611 1 + d / d ? i k + 8 i k ) l

= - ( Y I P ; + + v ,P:+) + fC6,l(Yl + P:*) + ?I, + aia = - 2P, P, + W I , Pi + PI, + PI P,) + = P,@,, - P,) + M I , + O(N - ‘1,

- ‘1

where the third equality uses (3.35). We also have

(3.54) NECQ;, - Q,,] = N E [ P ; , - Pi,] + O(N -- ’)

= PI, - Yl , + O w - ’ )

= - Q l , + O ( N - ’ ) ,

so since IP;, - Pl,l 5 N - ’ with probability one, the conditions (3.12H3.16) of Theorem 3.S are satisfied with cN = N- ’ and 6, = N-’ (recall (3.27)). Thus, the theorem is applicable to this model as well.

4. INFINITELY-MANY-ALLELE MODELS

In the absence of selection, the Wright-Fisher model (defined by (1.12). (1.6) with p: = p k , and (1.5)) can be described as follows. Each of the 2N genes in generation k + 1 selects a “parent” gene at random (with replacement) from generation k. If the “parent” gene is of allelic type A,, then its “offspring” gene is of allelic type A, with probability u$.

In this section we consider a generalization of this model, as well as its diffusion limit. Let E be a compact metric space. E is the space of “types.” For each positive integer M, let PM(x, F) be a transition function on E x g ( E ) , and define a Markov chain { YM(k) , k = 0, 1, . ..} in EM = E x * . . x E (M factors) as follows, where Y?(k) represents the type of the ith individual in generation k. Each of the M individuals in generation k + 1 selects a parent at random (with replacement) from generation k. If the parent is of type x, then its offspring’s type belongs to r with probability P,(x, I-). In particular,

436 CENmC MOOELS

if jl,. . . , j l E { 1, . . . , M} are distinct andf,, ...,/I E WE), since the components of YM(k + 1) are conditionally independent given YM(k).

Observe that the process

has sample paths in DerE,[O, a). Our first result gives conditions under which there exists a Markov process X with sample paths in D,,,,[O, 00) such that X " * X .

Suppose that B is a linear operator on C(E), and let

and let

4.1 Theorem Suppose that the linear operator E on C(E) is dissipative, 0 ( B ) is dense in C(€), and the closure of B (which is single-valued) contains (1,O). Then, for each 3 E P ( g ( E ) ) , there exists a solution of the DI(,,[O, 00) martingale problem for (A, ?). If the closure of B generates a Feller semigroup on C(E), then the martingale problem for A is well-posed.

For M = 1, 2, . .., define X H in terms of PM(x, r) as in (4.2), and define QM

and B M on B(E) by

(4.6) = ~f~.Y)PI(+ dY), EM = M(QM - 1).

If the closure of B generates a Feller semigroup on C(E), if .

(4.7) B c ex-lim EM, M-CO

and if X is a solution of the D,, , [O, 00) martingale problem for A, then XM(0) X(0) in 9 ( E ) implies X M * X in DB,E,[O, a).

4. INFINITELY-MANY-ALLELE MODELS 437

Proof. Under the first set of hypotheses on B, Lemma 5.3 of Chapter 4 implies the existence of a sequence of transition functions P,(x, r) on E x g ( E ) such that the operators BMr defined by (4.6). satisfy limM-,m B M / = Bf for all/€ 9 ( B ) . In particular, (4.7) holds.

Hence it suffices to prove existence of solutions of the martingale problem for A assuming that Q(B) is dense in C(E) and (4.7) holds. Let rp = n:= , ( A , .) E 9, choose f y, . .., f r E B(E) such that f y ~ J and B, /y-+Bh for i = 1, ..., k , and put rpM = fl:-l(/y, .). Given p = MAIZEI d,,,where(x ,,..., xM)eEM,wehave

where the factor M ! / ( M - k ) ! is the number of ways of selecting jl, . . . , j k so that they are all distinct, and M ! / ( M - k + l ) ! is the number of . . selectingj,, ..., j , so that j r = j , (I, M fixed) but they are otherwise Hence

ways of distinct.

+ ~ . - k + l M ! (M - k + I ) !

and the convergence is uniform in p of the form p = M - I Lt1 S,, where (x,, . . . , xu) E EM. Here we are using the fact that, since 9 ( B ) is dense in C(E), QM/- f for every /E C(E). As in Remark 5.2 of Chapter 4, it follows from Theorems 9.1 and 9.4, both of Chapter 3, that {X"} is relatively compact. As in Lemma 5.1 of Chapter 4 we conclude that for each ir E B(WE)), there exists a solution of the martingale problem for (A, 8).

Fix 0 E 9(9 (E) ) . To complete the proof, it will suffice by Corollary 8.17 of Chapter 4 to show that the martingale problem for (A, 3) has a unique solution, assuming that B generates a Feller semigroup {S(c)} on C(E). Let X be a solution of the martingale problem for (A, ir). By Corollary 3.7 of Chapter 4, X has a modification X* with sample paths in D9,a[0, a). Let fl, . . . ,ff Q 9(@. Then

Moreover, (4.12) holds for allf,, . . . , f L E C(E) since 9 ( B ) is dense in C(E). Let Y be another solution of the martingale problem for (A, 5). and put

(4.13) P d U ) = 1-1

Then, since (4.12) holds with X replaced by Y,

(4.14)

We conclude that pk(u) = 0 for all k z 1 and u z 0, and hence that X and Y have the same one-dimensional distributions. Uniqueness then follows from

0 Theorem 4.2 of Chapter 4.

The process X of Theorem 4.1 is therefore characterized by its type space E and the linear operator B on C(E). Let E,, E,, ... and E be compact metric spaces. For n = I , 2, ..., let q,,: En-+ E be continuous, and define n,: C(E)-+ C(E,) by n n S = / o q., 4.: 9 ( E , , b 9(E) by 4,p = p s i I , and fin: C M E ) ) -+ C(g(En)) by fin S = S 0 it..

4.2 Proposition Let B , , B z , ... and B be linear operators on C(E,), C(E,), ... and C(E), respectively, satisfying the conditions in the first sentence of Theorem 4.1. Define A , , A z , ... and A in terms of E , , E , , ... and E and El, 8,. . .. and B as in (4.3H4.5). For n = I, 2, . . ., let X , be a solution of the D,,,,[O, a)) martingale problem for A,,. If the closure of B generates a Feller semigroup on C(E), if

(4. IS) B c ex-lim B, (with respect to [n,)), r - m

and if X is a solution of the D,,E,[O, OD) martingale problem for A, then fi,(X,(O)) - X(0) in P(E) implies fin 0 X, * X in DelE,[0, 00).

Proof. By (4. I5),

(4.16) A c ex-lim A, (with respect to {fin]), n- m

so the result follows from Corollary 8.16 of Chapter 4. 0

We give two examples of Proposition 4.2. In both, En is a subset of E, q,, is an inclusion map, and hence 4. can be regarded as an inclusion map (that is, elements of HE,) can be regarded as belonging to 9 ( E ) ) . With this understanding. we can suppress the notation q, and 6,.

44f) GENFUC MOOELS

4.3 Example For n = I , 2, . , . let En = {k/& k E Z} u {A} (the one-point compactification), define B, to be the bounded linear operator on C(E,) given by

and B, f ( A ) = 0, where o2 2 0, and let X , be a solution of the D,,Em,[O, 00) martingale problem for A, (defined as in (4.3H4.5)). XI is known as the Ohta-Kimura model.

Let E = BB u (A} (the one-point compactification), define B to be the linear operator on C(E) given by

(4.18) BJX) = ha’J.(X)

and B’(A) = 0, where 9 ( B ) = {/E C(E): (f -f(A))In E C:(R)}, and let X be a solution of the o@(E)[o, 00) martingale problem for A (defined by (4.3x4.5)). X is known as the Fleming-Viot model.

By Proposition 1.1 of Chapter 5 and Proposition 4.2 of this chapter, X,(O)-X(0) in P(E) implies X , s X in D,,E,[O, a). (Recall that we are regarding B(E,) as a subset of iP(E).) The use of one-point compactifications here is only so that Theorem 4.1 and Proposition 4.2 will apply. It is easy to see that, for example, P{X(O)(R) = I } = 1 implies P(X(r)(W) = I for all t > O } = 1. 0

4.4 Example For n = 2, 3, . .., let En = {l/n, 2/n, ..., I}, define B, to be the bounded linear operator on C(E,) given by

(4.19)

where 0 > 0, and let X , be a solution of the martingale problem for A, (defined as in (4.3H4.5)). Observe that X,(t) = I p,(t)d,,,, where (pl(r), . . . , p,- , (r)) is the diffusion process of Section 1 with E pl] = 8/2 for i = I, . . . , r, prl ( i # j ) independent of i and j , and tr,, = 0 for i, j = 1, . . . , r. Thus, X , could be called the neutral r-allele model with uniform mutation. (The term “neutral” refers to the lack of selection.)

Let E = [O, 13, define B to be the bounded linear operator on C[O, 13 given by

(4.20)

where 1 denotes Lebesgue measure on [O, 13, and let X be a solution of the Dllo, 1110, 00) martingale problem for A (defined by (4.3H4.5)). We call X the infinitely-many-neutral-alleles model with uniform mutation.

4. INFIMTELY-MANY-AUELE MODELS 441

By Proposition 4.2, X,(O)*X(O) in 9[0, I ] implies X , * X in Dele* ,,LO. 00). (Again, we are regarding 9YEJ as a subset of P[O, 13.) Of course, with E = [O, I ] and

(4.21) M 2 8,

in Theorem 4.1, we have X M ( 0 ) * X ( O ) in 9[0, I ] implies X ' - X in Dsl0, , , [O, 00). Thus, X can be thought of as either a limit in distribution of certain (n - 1)-dimensional diffusions as n-+ 00, or as a limit in distribution of

0 a certain sequence of infinite-dimensional Markov chains.

The remainder of this section is devoted to a more detailed examination of the infinitely-many-neutral-alleles model with uniform mutation.

4.5 Theorem Given 3 E 9(8[0, I]), let X be as in Example 4.4 with initial distribution 3. (In other words, X is the process of Theorem 4.1 with E = [O, I]. with B defined on C[O, 1) by B/= +@(J A) - j ) , where 0 > 0 and R is Lebesgue measure on [O, I], and with initial distribution t.) Then almost all sample paths of X belong to CJl0, 11[0, 00). and

(4.22) P { X ( r ) E 9J0, 13 for all t > 0} = 1.

where PJO. I ] denotes the set of purely atomic Bore1 probability measures on co, 11.

Proof. Using the notation of Example 4.4, let X, have initial distribution 8, E S(B(E,)), where the sequence IF,) is chosen so that i;, * 3 on 9[0, I]. Then X, * X in Dafo, , , [O, a), and since Cslo. ,,[O, 00) is a closed subset of Dale, , , [O, 001, we have

I

(4.23) 1 = lim P{xn E Cslo, IJO, 0 0 ) } 5 P ( X E Cslo. IICo, 0 0 ) ) n - m

by Theorem 3.1 of Chapter 3.

forJE CCO, I], The proof of the second assertion is more complicated. Observe first that

is a continuous, square-integrable martingale, and (see Problem 29 of Chapter 2) its increasing process has the form

f r

(4.25)

Consequently, if y 2 2 andf, g E C[O, 13 with/; g L 0,Itd's formula (Theorem 2.9 of Chapter 5) implies that

is a continuous, square-integrable martingale, and

(Note that (., .) is used in two different ways here.) Let us define cp.,,: P[O, 1]-+[0, oo)foreachy>Oandn= 1.2, ... by

It follows that, for each y 2 2 and n 5: 1,2, .. . ,

is a continuous, square-integrable martingale with increasing process

in fact, this holds for each y > I as can be seen by approximating the function x y by the C2[0, 00) function (x + E)? Defining cpy: S[O, 11- [0, a] for each Y > O b Y

(4.3 1)

4. INHMTELY-MANY-ALLELE MODELS 443

we have bp-Jim,,,, q,,. , -- 'pv for each y 2 I, while q,,, P v , as n P a, for 0 c y 1,

(4.32) z,(t) = cp,(W)) - cp,(X(O))

is a continuous, square-integrable martingale with increasing process

(4.33)

here we are using limn,m E[Z, , ,(t)2] = limn-, € [ I , . Jt) ] = E[I,(t)J and the monotone convergence theorem to show that, when 1 < y < 2,

(4.34) E [ ( l c p v - I(X(s)) ds)'] < 00, t 2 0.

Letting qI +(p) = bp-lim,,, + cpv(cr) = ~ o r x s l ~ ( { x ) ) , we have

( I - cp, +)(X(s)) ds 1 0 = lim E[Z,(r) - Z2(t)] = E [sb (4.35) y - 2 +

for all f 2 0, so P { X ( r ) E 9J0, I ] for almost every t > 0 ) = 1. To remove the word "almost," observe that

(4.36) lim E [ Z y ( t ) 2 ] = lim ECl,(r)] y + l + ,-.It

= E[[vl + ( I - cp, + X W ) ds = 0 1 for each r 2 0 by (4.33) and (4.35). Fix to > 0. By Doob's martingale inequality, supo,,,,,(Z,(r)~-+O in probability as y-+ I + , so there exists a sequence yn-+ I + such that S U ~ ~ , ~ ~ ~ ~ I Z , ( I ) ~ - , O ~ . ~ . Letting

(4.37)

we obtain from (4.32) and (4.35) that, almost surely,

(4.38)

Since q(t) is nondecreasing in t , we conclude that P ( X ( t ) E 9J0, 13 for all t > 0 ) = 1, as required. 0

( P I t(X(0) - C P I t(X(0)) - '10) + iot = 0, 0 s t 5 t" .

4.6 Theorem The measure-valued diffusions X,, , n = 2.3, . , . , and X defined as in Example 4.4, have unique stationary distributions f i n , n 5 2, 3, . . . , and fi, respectively. In fact, @,, is the distribution of c;= I ClS,,., where (<!, . . . , C:) has a symmetric Dirichlet distribution with parameter O/(n - I ) (defined below).

411 CEMnCMOOLLS

Moreover, there exist random variables C I 2 such that

2 * - 2 0 with cp", C1 = I

(4.39) (<:I) 9 ' P t y k ) ) I* (€1, * * # t k )

as n-+ a0 for each k 2 1, where ..., &, denote the descending order statistics of t!, , . . , t:. Finally, ji is the distribution of 1: I t1 d,, , where uI, u 2 , . . . is a sequence of independent, uniformly distributed random variables on [O, I], independent of tl, &, . .. .

Proof. Fix n 2.2. Let En = {l/n, 2/n, ..., 1) and define!,, ..., f. E C(E,) by fJj/n) = d,, , Let (t;, . . . , 4:) have a symmetric Dirichlet distribution with parameter E, = 0/(n - I), that is, (<;, . . . , (:- ,) is a K-valued random variable (recall (1.16) with r = n ) with Lebesgue density r(n&a)r(&a)-n(pl - - . pn)4-l, and (; = 1 - z:,' el. Let 5, E P(P(E,)) be the distribution of c;-l C;S,,,. To show that 3, is a stationary distribution for X,, it suffices by Theorem 9.17 of Chapter 4 to show that

(4.40) 5 @ ( E d .a( i- ir 1 (J p)m')ia(dfi) * 0

for all integers M I , ..., m, L 0, where G, is as in (4.4) with B, given by (4.19). (Actually, this can be proved without the aid of Theorem 9.17 of Chapter 4 by checking that the span of the functions within parentheses in (4.40) forms a core.)

But with I m I = m, + - - + m,, the left side of (4.40) becomes

(4.41)

4. INFIMTELV-MANV-ALLELE MODELS U S

and this is zero because

(4.42)

and r(u) = (u - I)r(u - I ) if u > 1. As for uniqueness, suppose fin is a stationary distribution for X , , and note that the left side of (4.41) with CV, replaced by fin is zero. Inducting on the degree of H7,= I < A , p)"' (namely, I m 1 ), we find that I n;=, ( A , p)""fi,(dp) is uniquely determined for all M I , ..., m, 2 0. Hence ji, is uniquely determined.

By Theorem 9.3 of Chapter 4, X has a stationary distribution f i , Noting that

(4.43)

for all k 2 1 and f,, . . . , fk E C[O, 11, we obtain the uniqueness of f i as above, except here we induct on k. It follows from Theorem 9.12 of Chapter 4 that fin sg fi on 9[0, I].

Theorem 4.5 immediately implies that fi(9J0, I]) = 1. We leave it to the reader to check that therefore there exist random variables {, 2 <, 2 * * * 2 0 with Cp", [, = I and u,, u 2 , . . . with distinct values in [O, I ] such that fi is the distribution of I (,&, . The assertion that (4.39) holds says simply that the joint fin-distribution of the sizes of the k largest atoms converges to the joint ,&distribution. Unfortunately, this cannot be deduced merely from the fact that fin =. fi on P[O, I]. However, by giving a stronger topology to 9[0, I], we can obtain the desired conclusion.

We define the metric p* on P[O, I ] as follows: given p. v E S [ O , I], let F,,, F , be the corresponding (right-continuous) cumulative distribution functions, and put

(4.44)

where dDI0, ,, denotes a metric that induces the Skorohod topology on D[O, I ] (see Billingsley (1968)). The separability of ( S [ O , I], p*) follows as does the separability of D[O, I]. We note that I t;d,,,, n = 2, 3, ..., and c,?, &dU, are (P[O, I], p+)-valued random variables, so we regard their distributions fin, n = 2.3,. . . , and f i as belonging to P(9[0, I]. p*). We claim that

P*(lb v ) = dwo, I I(F, * F") t

(4.45)

Letting

(4.46) FAt) = €7, F(t) = 1 €,, 0 5 t 5 1,

we see from the definition of p+ that it sufices to show that F, - F in D[O, I]. We verify this using Theorem 15.6 of Billingsley (l968), which is the analogue for 010, 1 J of Theorem 8.8 of Chapter 3.

fin =. 6 on (9[0, 13, P*) .

I lns i U i S I

446 GENETIC MODUS

Let C = u,t I {t f: [O, I]: P { q = t } > 0). Then C is at most countable, and for r , , .. ., r, E [O, 1) - C, the function p--+(p([O, t , ] ) , ..., p([O, r J ) ) is $-as. continuous on (SCO, 13, p) ( p being the Prohorov metric), so

(4.47) (Fn(r 1)s . . J‘Atd) 3+ (F(t I), - - - 3 F(tk))

by Corollary 1.9 of Chapter 3. In particular, if t E [0, 11 - C, then

(4.48) E[F( t ) ] = lim E[F,(t)]

= lim &[<: + -. + <!,,,,I n-m

n-m

so E [ F ( t ) - F(t - )] = 0, and hence C = @. it follows that (4.47) holds for all t l , . . ., t, E [0, 1). Finally, let 0 5 t l s t s rz s 1. Then for n = 1, 2,. . ., (4.49) EC(Fn(t) - Fn(r1))’(Fn(tz) - Fn(t))’I

= EC(Crnt, l+ 1 + * + C[nr~)2(<~m~+ I + * * * + Cr,,t$I

= EI(z;)z(z;)~I, where (27, Z;) is a K-valued random variable (recall (1.16) with r = 3) with Lebesgue density T(a, + /In + rn){r(an)r(~n)r(yn)}-lp~-lpg”-lpg-l and (an, P n , 7.1 = ((CntI - Cnt,J)&n , (Cntzl - Cntl)~, , (n - Cntd f Cnt,])~.). Hence (4.49) becomes

(4.50) an(an + I ) P n ( P n + 1) (an + P n + Yn) * * * (am + 8. + Yn + 3)

~ (Cntl ; rnt11>( Cnt2l ; Cnrl)

s ( t2 - t,) ’ .

We conclude that F , =+ F in D[O, I], and hence (4.45) holds. Let us say that x B D[O, J] has a jump of size S > 0 at a location f f: [O, 17

if I x( t ) - x(t - ) I = 6. For each x E D[O, l] and I 2 1, we define six) and IAx) to be the size and location of the ith largest jump of x. I f six) = s,+,(x), we adopt the convention that IXx) < IrC1(x) (that is, ties are labeled from left to right). If x has only k jumps, we define SAX) = IAx) = 0 for each i > k. We leave it to the reader to check that sl, s2 , . . . and 11, f 2 , . . . are Bore1 measurable on DCO, J]. Suppose {x,,} t 010, 17, x c DCO, J], and dmo, &,,, x ) 3 0. Then

(4.5 I ) (s1(xn), * * - I dxn))-+ (sI(xh sk(X))

and, if s,(x) > sz(x) > - -, then (4.52) ( s ~ ( x ~ ) * - . - I s&), l l (xn), -. - 9 4(xn))+ (sI(x), -. - 9 SAX), I,(xX * 3 C(x)).

It follows from the definition of p* that (4.53) (sl(FJ, * * * 9 Sk(F,))

4. INFINITNY-MANV-ALLEL€ MOMS 417

is a p*-continuous function of I( E S[O, I], where F, is as in (4.44). Now since the $,-distribution of (4.53) is the distribution of . . . , <&) for each n 2 k, and since the ji-distribution of (4.53) is the distribution of (el. ..., (k), we obtain (4.39) from (4.45).

We leave it as a problem to show that

(4.54) P ( { , > €2 > ...) = I

(Problem 12). It follows that

(4.55)

is a fi-a.s. p*-continuous function of p E 9[0, I]. Now the fin-distribution of (4.55) is the distribution of

(4.56) (<;I)* - * - t t ; k ) , u!, * * . . u;)

for each n 2 k, where (nu;, . . ., nu,") is independent of (I$, . . . , {,") and takes on each of the permutations of (1, 2, . . . , n) with probability l / n ! , By Corollary 1.9 of Chapter 3, (4.56) converges in distribution as n -+ 00 to the ji-distribution of (4.53, that is, to the distribution of (tl, .... ( k , ul, ..., uk). This allows us to conclude that u l r u 2 , . . . is a sequence of independent, uniformly distributed random variables on [O, I], independent of C I , (,, . . . . 0

We close this section with a derivation of Ewens' sampling formula. Given a positive integer r, a vector p = (PI. . . . , /3,) belonging to the finite set

(4.57) r

a E ( E + y :

and 3 E S(S,[O, 1)). let P(Q, 3) denote the probability that in a random sample of size r from a population whose "type" frequencies are random and distributed according to 3, p, " types " are represented j times ( j = 1, . . . , r ) .

4.7 Theorem Let ji be as in Theorem 4.6, let r 2 I, and let p E rr .Then

(4.58)

Proof. Observe that for each B E Sr(S,[O, I I),

(4.59)

and

(4.60)

118 CEMlCMOOCLS

where the sum ranges over all sequences (m,, m 2 , . . .) of nonnegative integers for which cz, mi = r and /I, is the cardinality of { i 2 1: m, = j } ( j = I , .. ., r). Denote (4.60) by PAP). Then 'p, is lower semicontinuous with respect to p* and

(4.61) w r,

implying that 'pp = 1 - xaar,-(,,)rpa is also upper semicontinuous with respect to p+, hence p+-continuous. We conclude that

c Pa@) = (SI(Q + S,(F,) + * - 9' = 1,

(4.62)

The proof is completed by showing that the left side of (4.62) equals the right side of (4.58). That is,

(4.63)

where the sums are as in (4.60); here we are using r(e,,) = r( 1 + &,,)/&,. 0

5.

1.

2.

PROBLEMS

Show that (1.23), (1.26), and (1.27) imply (1.22).

Let X be the diffusion process in K of Section 1 in the special case in which prJ r: y, > 0 for i, j = 1, ..., r. Show that the measure p E qK). defined by

5. PROILEMS 449

for some constant /.I > 0, is a stationary distribution for X. (This generalizes parts of Theorems 2. I and 4.6.)

3. Let X be as in Theorem 2.4. Show that r , defined by (2.56), is continuous as. with respect to the distribution of X for 0 c E < 4. Let X be the diffusion process in K of Section I in the special case in which p,, = y > 0 for i = I, . . ., r - I , p,, = 0 otherwise, and u,, = 0 for i , j = 1, ..., r. Let t = inf ( t z 0: minI,,,,-,Xi(r) = 0) . I f p E K and P ( X ( 0 ) = p } = 1,show that,for i = 1, ..., r - I ,

4.

(5.2) P ( X M = 0 ) = 1 - pi{ (Pi + P,)-' 1 S I s r - I

i+ 1

- 1 ( p r + p , $ P & ) - l + . . . + ( - l ) ' - l ( l - - p p I

I S j < k S f - - 1 I , * k * d

5. Put I = { I , ..., r ) , and let X be the diffusion process in E = [0, I]' with generator A = {(f, GI):/€ Cz(E)} , where

(p,,),. ,. ;is the infinitesimal matrix of an irreducible jump Markov process in I with stationary distribution and d is real. (Specifically, pi, 2 0 for i # j , there does not exist a nonempty proper subset J of I with p i , = O f o r a l l i ~ J a n d j ~ J , ~ , > O f o r e a c h i E I , ~ , E , ~ , = l , a n d zrs, ~ , p , ) = 0 for eachj E I.) (a) Formulate a geographically structured Wright-Fisher model of

which X is the diffusion limit, proving an appropriate limit theorem (cf. Problem 7).

(b) Let T = inf ( t 2 0: X ( t ) = (0, ..., 0) or X ( f ) = (1, ..., I )} . Show that E[TJ < 00, regardless of what the initial distribution of X may be.

(c) If p E E and P{X(O) = p ) = 1, show that

6. Construct a sequence of diffusion processes XN in [O. I]. and a diffusion X in [0, I ] with the following properties: 0 and I are absorbing boundaries for X N and X , and X N =- X in D,,,, lr[O, a); but, defining { by (2.20) with F = (0, I } and T~ and r by (2.21). T~ fails to converge in distribution to ?.

7. Apply Theorem 3.5 to Nagylaki's (1980) geographically structured Wright-Fisher model. In fact, this is already done in the given reference,

450 GENETIC MODELS

8.

9.

10.

11.

12.

13.

14.

so the problem consists merely of verifying the analysis appearing there and checking the technical conditions.

Apply Theorem 3.5 to the dioecious multinomial-sampling model described by Watterson (1964).

Hint: Rather than numbering the genotypes arbitrarily, it simplifies matters to let fij) be the frequency of A,A, (i s j ) in sex s(s = 1, 2). This suggests, incidentally, that the general case of r alleles is no more dificult than the special case r = 2. Finally, we remark that the assumption that mutation rates and selection intensities are equal in the two sexes is unnecessarily restrictive.

Apply Theorem 3.5 to the dioecious overlapping-generation model described by Watterson (1964).

Karlin and Levikson (1974) state some results concerning diffusion limits of genetic models with random selection intensities. Prove these results. Hint: As a first step, one must specify the discrete models precisely.

Let X , (respectively, X) be the Ohta-Kimura model (respectively, the Fleming-Viot model) of Example 4.3, regarded as taking values in 9(Z) (respectively, 9(R)). Show that XI (respectively, X) has no stationary distribution.

Let { I 2 cL2 2 * * . be as in Theorem 4.6. Show that P{tl > tz > - a * ) = 1.

Let tr 2 tz 2 6 * * be as in Theorem 4.6. Consider an inhomogeneous Poisson process on (0, a) with rate function p(x) = Ox-’e-’. In particular, the number of points in the interval (a, 6) is Poisson distributed with parameter Jt p(x) dx. Because Ax) dx < 00 (= a) if a > 0 (= 0), the points of the Poisson process can be labeled as ql > qz > * - * . Moreover, c,Z, ql has expectation xp(x) dx = 0, and is therefore finite as. Show that (C1, C z , ...) has the same distribution as

(5.5)

Let X be the stationary infinitely-many-neutral-alleles model with uniform mutation (see Theorem 4.6), with its time-parameter set extended to (- 0O.00).

(a) Show that {X(t ) , -m < t < 00) and { X ( - t ) , -a < t < a) induce the same distribution on Celo, - 00, a). (Because of this, X is said to be reuersible.)

Using (a), show that the probability that the most frequent allele (or “type”) at time 0, say, is oldest equals the probability that the most frequent allele at time 0 will survive the longest.

(b)

6. NOTES 451

(c) Show that the second probability in (b) is just E[<J where <, is as in Theorem 4.6. (See Watterson and Guess (1977) for an evaluation of this expectation.)

6. NOTES

The best general reference on mathematical population genetics is Ewens (1979). Also useful is Kingman (1980).

The genetic model described in Section 1 is a variation of a model of Moran (1958~) due to Ethier and Nagylaki (1980). The Wright-Fisher model was formulated implicitly by Fisher (1922) and explicitly by Wright (I93 I). Various versions of Theorem 1 . 1 have been obtained by various authors. Trotter (1958) treated the neutral diallelic case, Norman (1972) the general diallelic case, Littler (1972) the neutral multi-allelic case, and Sato (1976) the general multi-allelic case.

The proof of Lemma 2.1 follows Norman (1975b). Theorem 2.4 is essentially from Ethier (1979), but a special case had earlier been obtained by Guess (1973). Corollary 2.7 is due to Norman (1972).

Section 3 comes from Ethier and Nagylaki (1980). Earlier work on diffusion approximations of non-Markovian models includes that of Watterson (1962) and Norman (197Sa). Example 3.8, as noted above, is similar to a model of Moran (1958~). Example 3.9 is essentially due to Moran (1958a, b).

Theorem 4.1 is due to Kurtz (1981a). The characterization of X had earlier been obtained in certain cases by Fleming and Viot (1979). The processes of Example 4.3 are those of Ohta and Kimura (1973) and Fleming and Viot (1979). Example 4.4 was motivated by Watterson (1976). but the model goes back to Kimura and Crow (1964). Theorem 4.5 is analogous to a result of Ethier and Kurtz (1981). The main conclusion of Theorem 4.6, namely (4.39), is due to Kingman (1975). Finally, Theorem 4.7 is Ewens’ (1972) sampling formula; our proof is based on Watterson (1976) and Kingman (1977).

Problem 2 is essentially Wright’s (1949) formula. The reader is referred to Shiga (1981) for uniqueness. Problem 4 comes from Littler and Good (1978). See Nagylaki (1982) for Problem 5(aMc). Problem I 1 (for X,) is due to Shiga (1982), who obtains much more general results. Problem 13 is a theorem of Kingman (1975). while Problem 14 is adapted from Watterson and Guess (1977).

11

By a population process we mean a stochastic model for a system involving a number of similar particles. We use the term “particles” broadly to include molecules in a chemical reaction model and infected individuals in an epidemic model. The branching and genetic models of the previous two chapters are examples of what we have in mind.

In this chapter we consider certain one-parameter families of processes that arise in a variety of applications. Section 1 gives examples that motivate the general formulation. Section 2 gives the basic law of large numbers and central limit theorem and Section 3 examines the corresponding diffusion approximation. Asymptotics for hitting distributions are considered in Section 4.

DENSITY DEPENDENT POPULATION PROCESSES

1. EXAMPLES

We are interested in certain families of jump Markov processes depending on a parameter that has different interpretations in different contexts, for example, total population size, area, or volume. We always denote this parameter by n. To motivate and identify the structure of these particular families, we give some examples: 452



1. EXAMPLES 453

A. logistic Growth

In this context we interpret n as the area of a region occupied by a certain population. If the population size is k, then the population density is k/n. For simplicity we assume births and deaths occur singly. The intensities for births and deaths should be approximately proportional to the population size. We assume, however, that crowding affects the birth and death rates, which therefore depend on the population density. Hence the intensities can be written

If we take A(x) E Q and p ( x ) = b + cx, we have a stochastic model analogous to the deterministic logistic model given by

( 1 .a k = (a - b)X - cxz. 0

8. Epidemics

Here we interpret n as the total population size, which remains constant. In the population at any given time there are a number of individuals i that are susceptible to a particular disease and a number of individuals j who have the disease and can pass it on. A susceptible individual encounters diseased individuals at a rate proportional to the fraction of the total population that is diseased. Consequently, the intensity for a new infection is

(1.3)

We assume diseased individuals recover and become immune independently of each other, which leads to the assumption

The analogous deterministic model in this case is

k, = - A X , x,, k, = 1x1 x2 - px,. 0

454 DENSITY DHENOWT H)PUUTION PUOCIESSES

C. Chemical Reactions

We now interpret n as the volume of a chemical system containing d chemical reactants R , , R , , . . . , R, undergoing r chemical reactions

1 - 1 . 2 ,..., r,

that is, for example, when the jth reaction occurs in the forward direction, b,, molecules of reactant R , , b,, molecules of reactant R , , and so on, react to form c,, molecules of R, , cl, molecules of R , , and so on. Let b, = (b,,, b,,, . . . , bdj), c1 = (c,,, caj, . . . , c,,), and define

(1.7) 1

I b , I = b l , + b s i + b , , + . . . + b l J , xbJ= n x p j . I = 1

The stochastic analogue of the “law of mass action” suggests that the intensity for the occurrence of the forward reaction should be

where k = ( k , , k , , . . . , k,) are the numbers of molecules of the reactants. The intensity for the reverse reaction is

If we take as the state the numbers of molecules of the reactants, then the transition intensities become

and the analogous deterministic model is r

(1.11) *1 m= 1 ((ci, - bi,)rf, x’’ f (bi, - CU#, xc’) I- I

where

2. U W OF URGE NUMBERS AND CENTRAL LIMIT THEOREM 455

In the first two examples the transition intensities are of the form

(1.12)

In the last example, (1.12) is the correct form if cl, and h,, only assume the values 0 and 1, while in general we have

(1.13) 4:1:+r = . [ P I ( $ + 0 ( 3 ]

We consider families with transition intensities of the form (1.12) and observe that the results usually carry over to the more general form with little additional effort. To be precise, we assume we are given a collection of nonnegative functions

PI , 1 E E', defined on a subset E c Rd. Setting

(1.14) En = E n { n - ' k : k E Hd},

we require that x E En and f l l (x ) > 0 imply x + n - ' I E. Em. By a density dependent family corresponding to the PI we mean a sequence ( X , ) of jump Markov processes such that X, has state space €, and transition intensities

(1.15) q r ' y = nfln(y-r)(Xh X, Y E E n *

2. L A W OF LARGE NUMBERS A N D CENTRAL LIMIT THEOREM

By Theorem 4.1 of Chapter 6 we see that the Markov process gm with intensities qr)L+l = np,(k/n), satisfies, for t less than the first infinity ofjumps,

where the Setting

are independent standard Poisson processes.

456 DENSITY OePENDENT PORHATION PROtEssES

where q(u) = Ydu) - u is the Poisson process centered at its expectation. The state space for X, is E, given by (1.14), and the form of the generator for X, is

(2.4) An f(x) = C nB,(xKf(x + n- 'O -f(x)) I - c n/?,(xMf(n + n - ' I ) - f ( x ) - n - ' I Vf(x ) ) 1

+ F(x) * Vf(X),

Iim sup I n- l9;(nu)l = o

x E En.

Observing that

as., v 2 0, a-m Y S U

(2.5)

we have the following theorem.

2.1 Theorem Suppose that for each compact K c E,

and there exists M, > 0 such that

(2.7) x, Y E K.

Suppose X , satisfies (2.3), Iim,*- X,(O) 5 xo, and X satisfies

(2.3)

I F(4 - FWI s M ~ l x - Y I *

X ( t ) = x,, + p ( X ( s ) ) ds, t 2 0. 0

Then for every t 2 0,

lim sup I X,(s) - X(s)l = 0 a.s. n-m ~ S I

(2.9)

2.2 Remark Implicitly we are assuming global existence for k = F ( X ) , El X(0) = xo . Of course (2.7) guarantees uniqueness.

Proof. Since for fixed t 2 0 the validity of (2.9) depends only on the values of the in some small neighborhood of {X(s): s 5 r } , we may as well assume that f i l 3 supxrE PAX) satisfies

(2.10) c I1lSt < * and that there exists a fixed M > 0 such that

2. LAW OF LARGE NUMIERS AND CENTRAL L I N T THEOREM 457

Then

(2.12)

where the last inequality is term by term. Note that the process on the right is a process with independent increments, and the law of large numbers implies

(2.13) Iim 1 I I I n - ' ( x ( n j I t ) + nfiI r ) n - m I

=I 2 l / l l j f f 1

= 1 Iim I I 1 n - ' (Y,nflf t ) + nIr, t ) , I n - r m

that is, we can interchange the limit and the summation. But the term by term inequality in (2.12) implies we can interchange the limit and summation for the middle expression as well, and we conclude from (2 .5) that

(2.14) lim &,(I) = 0 a.s. n-+m

Now (2.1 I ) implies

and hence by Gronwall's inequality (Appendix 5).

and (2.9) follows. 0

Set Wj"'(u) = n' R(nu). The fact that Wr' * q , standard Brownian motion, immediately suggests a central limit theorem for the deviation of X , from X . Let V,(t) = &(X,,(t) - X ( f ) ) . Then

4% DENSITY DEPENDENT r o I u u T l O N PROCESSES

Observing that X , = X + n- ’”V, , (2.17) suggests the following limiting equation:

(2.18) V( t ) = V(0) + C 14 1

5 V(0) + U(r) + dF(X(s))V(s) ds, sb where aF(x) = ((a, FAX))).

Let 0 be the solution of the matrix equation

a - @(t, S ) = dF(X(t))co(r, s), at

Q(s, s) = 1. (2.19)

Then

(2.20) V(t) = uqc, O)V(O) + yf, s) d[l(s) l = @(t, O)V(O) + U(r) + @(t, s) dF(X(s))U(s) ds

rt sb

= @(t, OKV(0) + [l(t)) + @(t, S) dF(X(s))(U(s) - V(r)) ds. I Since U is Gaussian (in fact, a time-inhomogeneous Brownian motion), Y is Gaussian with mean @(r, O)V(O) (we assume V(0) is nonrandom) and covariance matrix

(2.21)

where

cov (W), W) = LA‘@& s)G(X(s))C@(r, s)IT ds,

(2.22) G(x) = C ilrj?I(x). 1

2.3 Theorem Suppose for each compact K c E,

(2.23)

and dF are continuous. Suppose X, satisfies (2.3), X satisfies - X), and Iim,-- K(0) = V(0) (V(0) constant). Then V, * V

where V is the solution of (2.18).

Proof. Comparing (2.17) to (2.18), set

(2.24)

3. DIFFUSION ACMOXIMATIONS 459

and

Theorem 2.1 implies supss, I E ~ S ) I -+ 0 as. and II, * V in DR,[O, a). But, as in (2.20),

(2.26) V.(t) = W, O)K(O) + v,(t) + + Wl, S) dF(X(s)MU,(s) + E,(s)) ds,

and V,

Jo Y by the continuous mapping theorem, Corollary 1.9 of Chapter 3 . 0

3. DIFFUSION APPROXIMATIONS

The basic implication of Theorem 2.3 is that X , can be approximated (in distribution) by 2, = X + n - - ’ ’ 2 V . An alternative approximation is the “diffusion approximation ” Z, , whose generator is obtained heuristically by expandingfin (2.4) in a Taylor series and dropping terms beyond the second order. This gives

The statement that 2, approximates X , is, of course, justified by the central limit theorem. No similar limit theorem can justify the statement that 2, approximates X,, since the 2, are not expressible in terms of any sort of limiting process. To overcome this problem we use the coupling theorem of Komlos, Major and Tusnady, Corollary 5.5 of Chapter 7, to obtain a direct comparison between X , and 2,.

Suppose the /I, are continuous and the solution of the martingale problem for (En , b,,,,) is unique. Then it follows from Theorem 5.1 of Chapter 6 that the solution can be obtained as a solution of

(3.2) Z,(t) = X,(O) +

where the 4 are independent standard Brownian motions. By Corollary 5.5 and Remark 5.4 both of Chapter 7, we can assume the existence of centered Poisson processes 8 such that

(3.3)

160 DENSITY KlPMATION PROCESSES

and for p, y > 0 there exist A, K, C > 0 such that

(3.4) I ?At) - H((t)I > C log n + x

Since the W, are independent, we can take the to be independent as well. Let X , satisfy (2.3) using the 8 constructed from the &. Then X, and Z, are defined on the same sample space and we can consider the difference IX,(r) - Z,(r) I . (Note that the pair ( X " , 2,) is not a Markov process even though

each component is.)

3.1 Theorem Lct X , X , , and 2, be as above, and assume limR-,m X,(O) = X(0). Fix E, T > 0, and set N, = {YE E: inflsT IX(f ) - yl s 6 ) . Let b, = supxeNa fl,(x) < 00 and suppose PI = 0 except for finitely many 1. Suppose M > 0 satisfies

Let 'I, 5: inf {t: X,(t) fi! N, or Z,(t) f NJ. (Note P{T, > T } - , 1.) Then for n 2 2 there is a random variable ri and positive constants I,, C,, and K, depending on T, on M, and on the 1,. but nor on n, such that

and

P{rr > C, + x} ?; K,n-l exp ( - l , ~ l / ~ - log n

Proof. Again we can assume 8, = supxes M x ) -z 00 and (3.5) and (3.6) are satisfied for all x, y c E. Under these hypotheses we can drop the 7, in (3.7).

By (3.4) there exist constants C:, K,' independent of n and nonnegative random variables Li , such that

(3.9)

and

(3.10)

3. DIFFUSION APPROXIMATIONS 461

Define

(3.1 I ) Al. ,(z) = SUP SUP I W ~ U + U ) - W,(U)( . u s n h r u j z l o g n

Let k, = [np, T / z log n] + 1. Then

(3.12) A l . n ( ~ ) S 3 SUP SUP I W ( ~ Z log II + U) - W,(&Z log n ) l , k s h v j z l o # r

and hence for z, c, x 2 0

(3.13) P{Al , .(z) L zl"(c log n + x))

I 1

Is k, P 3 sup I W(u)I 2 z'12(c log n + x)

5 2 k , P ( I W,(l)l 2 3(c log it + x)(log H ) - I ' ~ }

{ v s z I o # n

(c2 log n + 2cx -t .?/log n) 18

where a = supxro er*'2P( { Wdl)l 2 x). Since AIJz) is monotone in z ,

(3.14) hI., E sup ( z + I ) "zAl ,n(z ) Q 5 r z i n l i t

S sup m - 1 ' 2 A l , n ( n ~ ) I s m s n h T + I

for integer m. Therefore

(3.15) P ( A , , , > c log n + x}

(c' log n + 2cx + x2/log n) 18

5 (1 + nfll T)ak, exp

Since k, = O(n) there exist constants C: and K,?, independent of n, such that

(3.16) P{Alan > C: log n + x) s K:n-2 exp { -2.x -- 18 log n

Setting L: = (A,# ,, - C: log n) V 0, we have

(3.17) P { t t . > x) s K f n - . ~ exp {--Ax - L] . 18 log n

Taking the difference between (2.3) and (3.2) (recall only finitely many PI are nonzero),

4662 DENSITY DEPENDENT PWULATION PROCESSES

+ M I X,(S) - Z,(S) I ds. l Let y,,(f) = n I X,(r) - &(t) I /log n. Then

and setting 7, = y,,(t) we have

(3.20) i;. s eMT I

The inequality y 5 a + y'% implies y 5 2a + b'. Hence there is a constant CT such that

I= CT + L, r:. Since the sums in (3.21) are finite, (3.10) and (3.17) imply there exist constants K T , I T > 0 such that

(3.22) P{L, > x} s K , n - l exp - ITx l" - and (3.8) follows. 0

3. MFFWION AIIUOXIMATIONS 463

We now have two approximations for X,, namely 2, and 2,. The question arises as to which is “best.” The answer is that, at least asymptotically, for bounded time intervals they are essentially equivalent. In order to make this precise, we consider Z, and 2, as solutions of stochastic integral equations:

and

3.2 Theorem In addition to the assumptions of Theorem 3.1, suppose that the /I:” are continuously differentiable and that F is twice continuously differentiable in N,. Let Z, and 2, satisfy (3.23) and (3.24). Let n(X,(O) - X ( O ) ) - b 0. Then

C

(3.25) SUP I NZn(l) - Z n ( t ) ) - P(r) I 4 0, 1s.r

where Psatisfies

and V satisfies

(3.27) v(t) = I [fl,’{’(X(.) dW,(s) + [JF(X(s))V(s) ds. I

3.3 Remark The assumption that El > 0 for only finitely many I can be replaced by

Ill2@, < 00 and Il lZ sup IVf l , ! i2(~)12 < 00. 0 a c N,

Proof. Let I/, = J n c ~ , - X I . A simple argument gives

(3.28) E sup I U,(r) - V(r)12]-+ 0. Lr

By hypothesis n(X,(O) - X(O))+ 0. The second term on the right converges to the first term on the right of (3.26) by (3.28) of this chapter and (2.19) of Chapter 5. The limit in (3.28) also implies the last term in (3.29) converges to

CI the last term in (3.26), and the theorem follows.

By the construction in Theorem 3.1, for each T > 0, there is a constant Cr > 0 such that

n (3.30)

whereas by Theorem 3.2, for any sequence am-+ 00,

(3.3 1)

Since Bartfai’s theorem, Theorem 5.6 of Chapter 7, implies that (3.30) is best possible, at least in some cases, we see that asymptotically the two approximations are essentially the same.

} = O , CT log n

lim P sup I X J t ) - Z,(t)l > #-OD { ~ S T

lim P sup I Z,(z) - 2,(t)I > n a-m I 8 s T

4. HIlTlNC DISTRIBUTIONS

The time and location of the first exit of a process from a region are frequently of interest. The results of Section 2 give the asymptotic behavior for these quantities as well. We characterize the region of interest as the set on which a given function cp is positive.

4.1 Theorem Let Q be continuously differentiable on R”. Let X, and X satisfy (2.3) and (2.8). respectively, with Cp(X(0)) > 0, and suppose the conditions of Theorem 2.3 hold. Let

( 4 4 T, - inf { t : cp(X,(t)) s 0)

4. HllTMG DISTRIBUTIONS 465

and

(4.2) T = inf { I : rp(X(t)) s 0).

(4.4)

and

4.2 Remar& One example of interest is the number of susceptibles remaining in the population when the last infective is removed in the epidemic model described in Section 1. This situation is not covered directly by the theorem.

0 (In particular, T = a). However see Problem 5.

5. PROBLEMS

0

1.

2.

3.

4.

5.

Let X, be the logistic growth model described in Section 1. (a) Compute the parameters of the limiting Gaussian process V given by

Theorem 2.3. (b) Let 2, and 2, be the approximations of Xn discussed in Section 3.

Assuming Z,(O) = 2,(0) f 0, show that 2, eventually absorbs at zero, but that 2, is asymptotically stationary (and nondegenerate).

Consider the chemical reaction model for R , + R, e R, with parameters given by (1.10). (a) Compute the parameters of the limiting Gaussian process V given by

Theorem 2.3. (b) Let X(0) be the fixed point of the limiting deterministic model (so

X(t) = X ( 0 ) for all t 2 0). Then V , is a Markov process with stationary transition probabilities. Apply Theorem 9.14 of Chapter 4 to show that the stationary distribution for V, converges to the stationary distribution for V.

Use the fact that, under the assumptions of Theorem 2.1,

(5.1) xn( t ) - Xn(0) - ~ F ( x A ~ ) ) ds

is a local martingale and Gronwall's inequality to estimate P{sup,,, I X,,(s)

Under the hypotheses of Theorems 3.1 and 3.2, show that for any bounded U c [w" with smooth boundary,

- X(S)I 2 E } .

log n

Let X , = (S,, I,) be the epidemic model described in Section 1 and let X = (S, I ) denote the limiting deterministic model (S for susceptible, I for infectious). Let T,, - inf { t : r,,(t) = O}.

6. NOTES 167

(a) Show that if I(0) > 0, then I ( f ) > 0 for all r 2 0, but that liml+m l ( t ) =I 0 and S(o0) -= lim,-m S(r) exists.

(b) Show that if &(X,,(O) - X(0) ) converges, then ,~(S,,(T,,) - S(cx))) converges in distribution. Hint; Let y. satisfy

and show that X,,(y,,( .)) extends to a process satisfying the conditions of Theorem 4.1 with cp(x,, x,) = x2.

6. NOTES

Most of the material in this chapter is from Kurtz (1970b, 1971, 1978a). Norman (1 974) gives closely related results including conditions under which the convergence of V,(f) to V ( f ) is uniform for all t (see Problem 2). Barbour (1974, 1980) studies the same class of processes giving rates of convergence for the distributions of certain functionals of V, in the first paper and for the stationary distributions in the second. Berry-Esseen type results have been given by Allain (1976) and Alm (1978). Analogous results for models with age dependence have been given by Wang (1977).

Darden and Kurtz (1985) study the situation in which the limiting deterministic model has a stable fixed point, extending the uniformity results of Norman and obtaining asymptotic exponcntiality for the distribution of the exit time from a neighborhood of the stable fixed point.

12

The aim in this chapter is to study the asymptotic behavior of certain random evolutions as a small parameter tends to zero. We do not attempt to achieve the greatest generality, but only enough to be able to treat a variety of examples. Section 1 introduces the basic ideas and terminology in terms of perhaps the simplest example. Sections 2 and 3 consider the case in which the underlying process (or driving process) is Markovian and ergodic, while Section 4 requires it to be stationary and uniform mixing.

RANDOM EVOLUTIONS

1. INTRODUCTION

One of the simplest examples of a random evolution can be described as follows. Let N be a Poisson process with parameter 1, and fix a > 0. Given (x, y ) E R x { - 1, I}, define the pair of processes ( X , Y) by

X(t) and aY(t) represent the position and velocity at time t of a particle moving in one dimension at constant'speed a, but subject to reversals of direction at the jump times of N, given initial position x and velocity ay.

46a



1. INTRODUCTION 469

Let us first observe that (X, Y) is a Markov process in R x { - I , I ) . For if we define for each t 2 0 the linear operator T(t) on c(R x { - I , I } ) by

( 1 .a T ( t ) / ( x * y ) = ECS(X(l), Y(r))l.

ECJ(X(t) , Y ( t ) ) I~SYI = T(t - ~ ) l ( X ( S ) . Y ( 4 )

where (X, Y ) is given by ( l , l ) , then the Markov property of Y implies that

(1.3)

for all /E t?(R x { - 1, I } ) , (x, y ) E B8 x { - 1, I}. and t > s 2 0. I t follows easily that {T( t ) } is a Feller semigroup on C(R x { - 1, I}) and (X, Y) is a Markov process in R x { - 1, 1) corresponding to { T(t)} .

Clearly, however, X itself is non-Markovian. Nevertheless, while Y visits y E { - I, I } , X evolves according to the Feller semigroup { q(t)} on c(R) defined by

( 1.4) q(t) ( t ) / (x) =S(x 4- sty).

Consequently, letting r l , r , , ... denote the jump times of Y, the evolution of X over the time interval [s, t ] is described by the operator-valued random variable

(1.5) 9(~, t ) =: 7y(o,((r, V s ) A t - s)Tr,,, ,((t,V~)At - ( r , V . $ ) A t )

in the sense that

(1 4 for all/€ t ( R x { - I , 1)). The family ( S ( s , t). t 2 s 2 0 ) satisfies

(1.7) Y ( S , f)Y(t, u) = . T ( S , u),

and is therefore called a random eoolurion. Because Y "controls" the development of X, we occasionally refer to Y as the driving process and to X as the driven process. Observe that (1.3) and (1.6) specify the relationship between { T(t)} and {Y(t)). (Of course, in the special case of ( l . l ) , the left side of (1.6) can be replaced by j ( X ( r ) , Y ( f ) ) because .P: c 9:. In general, however, X need not evolve deterministically while Y visits y.) To determine the generator of the semigroup (T( i ) } , let

f~ C'*'(R x { - I , I ) ) and t , > t , 2 0. Then

ECS(X(t), W)) I 9," v c . 1 = m, t){.f( * 1 ~(O)j(X(S))

s s I s u,

470 RANDOM EVOLUTIONS

so by Lemma 3.4 of Chapter 4,

(1.10)

is an {.F:f-martingale, where

(1.1 I )

Identifying t(W x { - 1, I ) ) with c(R) x e((R), we can rewrite A as

A m Y) = ayf;(x, Y) + A{S(x, -Y) -m YN.

(1.12)

with 9 ( A ) = el(R) x e l ( R ) . Since W ( A ) c e(W) x C(R), it follows from the martingale property and the strong continuity of (T( t ) } that the generator of { T(r)} extends A. But by Problem 1 and Corollary 7.2, both of Chapter I, A generates a strongly continuous semigroup on e(R) x &(R). We conclude from Proposition 4.1 of Chapter 1 that A is precisely the generator of ( T ( f ) } .

This has an interesting consequence. Letfe C2(W) and define

(1.13)

for all (c, x) E [O, 00) x R. Then g and h belong to c2([0, 00) x R) and satisfy the system of partial differential equations

(1.14) g, = ag, - &.I - h),

h, = -ah, + A(g - h).

Letting u = t(g + h) and u = f(g - h), we have

(1.15) u, = au,,

u, = au, - 21v.

Hence u,, I: QU,, = au,, - 2Au, = au,, - (2A/u)u,, or

(1.16)

This is a hyperbolic equation known as the telegrapher's equation. Random evolutions can be used to represent the solutions of certain of these equations probabilistically, though that is not our concern here.

1. INTRODUCTION 471

In the context of the present example, we arc interested instead in the asymptotic distribution of X as a -+ cn and 1 --+ 'x) with a' = 1. Let 0 < E < I and observe that with a = I / E and I. = 1 / ~ * , (1.16) becomes

(1.17)

This suggests that as E - -+ 0, we should have X =. x f W in C,[0, m), where W is a standard one-dimensional Brownian motion. To make this prccise, let N be a Poisson process with parameter I . Given (x, y) E R x { - I , I } , define (Xc. Y c ) for 0 < E < I by

(1.18)

By the Markov property of Y E ,

(1.19) 2 c 'g E

M"f) = - YC(C) - - y + - Y'(s) n s 2

t & = X"C) - x + - Y " f ) - - y

2 2

is a zero-mean {9r}-martingale, and M'(t)' - f is also an {9:')-martingalc. I t follows immediately from the martingale central limit theorem (Theorem 1.4 of Chapter 7) that M e - W in D,[O, oo), hence by (1.19) and Problem 25 of Chapter 3, that X c - x + W in C,[O, 00).

There is an alternative proof of this result that generalizes much more readily. LetfE c"(W), and definex E 6 ' - " ( W x ( - I , I})for 0 < E Y ) = A x ) + YSW. 2

1 I 4 LCX, Y ) = - Y f ' W + fY2S"W - ; Y f ' ( 4 = 4f'W

E (1.21)

X', where 0

( I .22)

with initial smoot hness

for all (x, y) E W x { - I , I}. The desired conclusion now follows from Proposi- tion 1 . 1 of Chapter 5 and Corollary 8.7 of Chapter 4.

I t is the purpose of this chapter to obtain limit theorems of the above type under a variety of assumptions. More specifically. given F, G: Rd x E-- . Rd, a process Y with sample paths in D,[O, a), and x E Rd, wc consider thc solution

< E < I , of the differential equation

condition X"(0) = x. Of course, F and G must satisfy certain and growth assumptions in order for X" to be well defincd. In

472 RANDOM RlOLUTlONS

Section 2, we consider the case in which E is a compact metric space and the driving process Y is Markovian and ergodic. (This clearly generalizes (1.18).) In Scction 3, we allow E to be a locally compact, separable metric space. In Section 4, we again require that E be compact but allow Y to be stationary and uniform mixing instead of Markovian.

2. DRIVING PROCESS IN A COMPACT STATE SPACE

The argument following (1.20) provided the motivation for Corollary 7.8 of Chapter 1. We include essentially a restatement of that result in a form that is suitable for application to random evolutions with driving process in a compact state space.

First, however, we need to generalize the Riemann integral of Chapter 1, Section 1.

2.1 lemma Let E be a metric space, let L be a separable Banach space, and let p E B(E). Iffi E -+ L is Borel measurable and

then there exists a sequence { f n } of Borel measurable simple functions from E into L such that

The separability assumption on L is unnecessary if E is o-compact and f is continuous.

Proof. If L is separable, let {g,,} be dcnse in L; if E is a-compact and f is continuous, thenf(E) is a-compact, hence separable, so let {g,,) be dense in f(E). For m, n = 1, 2, .. . define A,,,,,, = {g Q L: Ilg - gall < I / m } - UtP: A&,,,, and

Then, letting Be,,,, = u:,., J-'(A&.,J, we have

1.

form, n = 1.2, .. ., and hence

Iim lim (2.5) m + m n-ap

DRIVING FROCESS IN A COMPACT STAT€ SPACE 473

We conclude that there exists {m, } such that (2.2) holds with f, = h,,. ,,,“. 0

Let E, L, and p be as in Lemma 2.1, and let f : E -+ L be a Borel measurable simple function, that is,

where B, , . . . , B, E .6(E) are disjoint, gl , . . . , g,, E L, and n 2 I . Then we define

(2.7)

More generally, supposefi E -+ L is Borel measurable and (2.1) holds. Let { fn)

be as in Lemma 2.1. Then we define the (Bochner) infegral off with respect ro p

by

It is easily checked that this limit exists and is independent of the choice of the approximating sequence { j,}.

In particular, if E is compact, L is arbitrary, and p E .P(E), then l f d p exists for all f belonging to C,(E), the space of continuous functions from E into I-. We note that C,(E) is a Banach space with norm IIIfIII = supy, IIf(y)ll.

2.2 Proposition Let E be a compact metric space, L a Banach space, P(t, y. r) a transition function on [O, 00) x E x B(E), and p E .P(E). Assume that the formula

(2.9)

defines a Feller semigroup (S( t ) } on C(E) satisfying

(2.10) lim 1 r e “S(t)g dt = 9 dp A - O + s

for all g E C(E), and let Bo denote its generator. Observe that (2.9) also defines a strongly continuous contraction semigroup {S( t ) } on C,(E), and let B denote its generator.

Let D be a dense subspace of L, and for each y E E, let n, and A, be linear operators on L with domains containing D such that the functions y - + Il, f

and y -+ A, f belong to C,( E ) for each f E D. Define linear operators n and A on C,(E) with

(2.1 1) 9(ll) = {SE C, (E) : / ( y ) E 9(n,) for every y E E,

and Y .+ n,(f(y)) belongs to C,(E)}

and

(2.12) 9 ( A ) = { f ~ C, (E) : f ( y ) E B(A,) for every y E E

and y+ A,(f(y)) belongs to C,(E)}

by ( ~ S ) ( Y ) = ~, (S(Y)) and (MWY) = A,.(J(y)). Let 9 be a subspace of C d E ) such that

c B c kd(ll) A 9 ( A ) n 9 ( B ) ,

and assume that, for 0 < E < 1, an extension of ( ( j ; nf+ & - ' A / + & - ' B f ) : f ~ 9) generates a strongly continuous contraction semigroup { '&(t)} on CL(E).

Suppose there is a linear operator Y on CL(&) such that A ~ E 9 ( Y ) and V A ~ E 93 for all / E D and BVq = -g for all g E 9 ( V ) . (Here and below, we identify elements of L with constant functions in C,(E).) Put

(2.14)

Then C is dissipative, and if c, which is single-valued, generates a strongly continuous contraction semigroup (T( t ) } on L, then, for each J E L, limc-o T , ( t ) / = T(t)ffor all r 2 0, uniformly on bounded intervals.

2.3 Remark Suppose that (2.10) can be strengthened to

(2.15)

By the uniform boundedness principle, there exists a constant M such that

and hence for each y E E there exists a finite signed Bore1 measure v(y, a ) such that

2. DRlVIffi PROCESS IN A COMPACT STATE SPACE 475

where the right side is defined using (2.8). If (AJHz)v( ., dz) E 9 for all J E D, then, by Remark 7.9(b) of Chapter 1, Y: { AJ: /E 0 ) -+ 9 is given by

(2.18) (VgKy) = g(Z)v(y, dz). 0 I Proof. We claim first that

is dense in C,(E). To see this, let c > 0 and choose y l , . . . , yn E E such that E = u;=I B(y,, 6). Let g l , ..., g, E C(E) be a partition of unity, that is, for i = 1, . . . , n, g, 2 0, supp g I c B ( y i , E ) , and c;= I gi = I (Rudin (1974), Theorem 2.13). GivenJ E C,(E), let 1; = I;= gr( - ) j ( y , ) . Then 1; belongs to (2.19) and

(2.20) Ill/; -JIII 5 SUP (IIJ(4 -J(Y)ll: r(x, Y ) < 4, where r is the metric for E. But the right side of (2.20) tends to zero as E -+ 0, so the claim is proved. Since 9 ( B , ) is dense in C(E) and D is dense in I., we also have 9, dense in C,(E).

I t follows that {S(f)} is strongly continuous on C,(E), (2.10) holds for all J E C,(E), and (2.16) implies (2.17). Note also that S(r): go-+ g o , so 9, is a core for B by Proposition 3.3 of Chapter I .

We apply Corollary 7.8 of Chapter 1 (see Remark 7.9(c) in that chapter) 0 with the roles of ll and A played by the restrictions of 17 and A to 9.

2.4 Theorem suppose that for each n 2 I there exists a constant M, for which

(2.21)

that GI, ..., G,, E C1*o(Rd x E), and that

Let E be a compact metric space, let F, G E Cm,(Wd x E), and

IF(x, f) - F(x', y)l I Mnlx - x' l , 1x1 v I x ' l 5 n, y E E,

(2.22)

Let {S( f ) } be a Feller semigroup on C(E), let p E P ( E ) , and assume that

(2.23)

Let B, denote the generator of {SCt)). G(x, y)p(dy) = 0 for all x E Rd and that there exists for each

y E E a finite, signed, Bore1 measure v(y, .) o n E such that the function H: Rd x E -+ Wd, defined by

(2.24)

lim 1 e-"'S(t)g dt = g dp, g E C(E). s." s 1-O+

Suppose that

WG Y ) = G(x, ZMY, dz) , s

476 RANDOM EVOlUnONS

satisfies, for i = 1, ..., d, H I E C'*o(Rd x E), H,(x, - ) E 9 ( E o ) for each x B R', and S,[HXx, . ) ] (y) = - CXx, y ) for all (x, y ) E tdd x E. Fix F~ E 9 ( E ) , and let Y be a Markov process corresponding to (Scr)) with sample paths in DEIO, a) and initial distribution po. Fix xo E Rd, and define X' for 0 < E < 1 to be the solution of the differential equation

dt (2.25)

with initial condition X'(0) = xo. Put

where

(2.27)

and

(2.28)

Then C is dissipative. Assume that c generates a Feller semigroup { T(r)} on (?(Rd), and let X be a Markov process corresponding to (T(t)} with sample paths in C,[O, 00) and initial distribution ~ 3 ~ ~ . Then X ' a X in C,[O, 00) as &--, 0.

a,l(x) = a x , Y ) H h Y)P(dY) + GhL JJ)H,(x. YIc((dY) I W ) = 1 F,(x* Y)F(dY) + G(x, Y ) v x m x , Y)AdY). s

2.5 Remark Suppose that (2.23) can be strengthened to

(2.29) SUP I WCdx, Il(y) - I Ax, z l W ) I dt < 00, 0 (I. y ) c R ' x E

g E t ( W d x E). Then u(y, dz) is as in (2.1 7) with L = t((R'). 0

Proof. We identify Ct,@,(E) with (?(El' x E) and apply Proposition 2.2 with

C,'(Iwd)), and

(2.30)

L = C'(R~),. D = c,~(R~), n, = F(., v) . v, A, = G(., JJ) . v. .qn,) = B(A,) =

9 = (/E C,'*O(W' x E ) : f ( x , .) E 9 ( E o ) for all x E W',

and (x, y)-+ E o [ f ( x , . ) ] (y ) belongs to e(Rd x E ) ) .

Clearly, 9 c D(n) n 9 ( A ) . We claim that 9 c O(l3). To see this, let (2.31) B = {(f, g) E e(Wd x E ) x e(R' x E):

J(x. .) E a(&,) for all x E Eld,

and d x , Y) = B,CS(x, -Il(y) for all (x, Y) E R' x E ) ,

2. ORIVING PROCESS IN A COMPACf SlATE SPACE 477

and observe that 8 is a dissipative linear extension of the generator B of (S(r)} on C(Rd x E), and hence b = B by Proposition 4.1 of Chapter 1.

Next, fix E E (0, I ) , and define the contraction semigroup { 7 ; ( r ) } on B(Rd x E ) by

(2.32)

where Y is a Markov process corresponding to (S( t ) } with sample paths in initial distribution d,, and X' satisfies the differential equation D,[O, 00) and

(2.25) with initial condition Xc(0) = x. The semigroup property follows from the identity

valid for all 1 > s 0 andj'E B(Rd x E) by the Markov property of Y and the fact that Xc(s + .) solves the dimerential equation (2.25) with ( X f ( 0 ) , V( - / tz2) )

replaced by (Xc(s), Y((s + .) /E')). We leave it to the reader to check that 7Jt): C(Wd x E)-+ C(Wd x E) for all f 2 0. Using (2.22) we conclude that 7 J r ) : C?((OBd x E)-+ C(Rd x E ) for every f 2 0. Let/E 93 and r 2 > I , 2 0. Then

and

so by Lemma 3.4 of Chapter 4,


is a martingale. It follows that { T(r)} is strongly continuous on 9, hence on C(Rd x E). We conclude therefore that the generator of { T , ( r ) } extends

We define V on 9 ( V ) = ( A ~ J E C,'(Rd)} by (VgKx, y ) = g(x, z)v(y, dz) and {(J ny-+ & - ' A l l + E-zSy-) :JE 9}.

note that V: 9 ( V ) - + 9 and

(2.37) ( ~ v m ( X l Y) = BoCWx, *) * V!(Xll(Y) = - w, y ) * V W ) = -AS(% v)

for allfE Cf(Rd) and (x, y ) E Rd x E. It is immediate that C, defined by (2.14), has the form (2.26H2.28). Under the assumptions of the theorem, we infer from Proposition 2.2 that, for each J E C(Rd), T&)l-+ T(r)f as E--, 0 for all r 2 0, uniformly on bounded intervals. By (2.33), (XI ( . ) , Y(./c2)) is a Markov process corresponding to ( T , ( r ) ) with sample paths in DRlxEIO, a) and initial distribution a,,, x po and therefore, by Corollary 8.7 of Chapter 4 and

0 Problem 25 of Chapter 3, Xt =. X in CR,[O, a)) as E-+ 0.

2.6 Example Let E be finite, and define

(2.38)

where Q = (qij)r, is an irreducible, infinitesimal matrix (i.e., qij 2 0 for all i # j, xj6 qil = 0 for all i E E, and there does not exist a nonempty, proper subset J of E such that qij = 0 for all i E J and j # J). Let p = (p i ) ieE denote the unique stationary distribution. It is well known that

(2.39) lim P(t, i, { j } ) = p j , i , j E E,

and (2.23) follows from this. By the existence of generalized inverses (Rao (1973), p. 25) and Lemma 7.3(d) of Chapter 1 , there exists a real matrix v = ( w f j ) f , j c E such that Q v l = -1 for all real column vectors 1 = for which A p = 0. I t follows that the function H of Theorem 2.4 is given by

1- m

(2.40)

Alternatively, using the fact that the convergence in (2.39) is exponentially fast (Doob (1953), Theorem VI.l.l), Remark 2.5 gives (2.40) with

(2.41) r m

This generalizes the example of Section 1.

2.7 Example Let E =: [O, I], and define

(2.42) Bo = ((9, +g'',: Y E c"0, 13, g'(0) = gY1) = O}.

0

3. DRIVING PROCESS IN A NONCOMPACT STATE SPACE 479

We claim that the Feller semigroup ( S ( r ) } on C[O, I ] generated by 8, (see Problem @a) in Chapter 1) satisfies

lim sup S(t)g(y) - g(z) dz = 0, g E C[O, I]. (2.43) 1-m O S Y S l I I ' l This follows from the fact that { S(!)) has the form

(2.44)

where

(2.45)

and p(f, y, z) = (2nf)-"' exp { - ( z - y) ' / 2 f f , together with the crude inequality

(2.46)

valid for 0 I y 5 1 and The function H of Theorem 2.4 can be defined by

(2.47) H(x, y) = - 2 [ l G ( x , w ) dw dz.

Note that H,(x, .) E CB(Bo) for each x E Rd and i = 1. . .., d since 0

Q m

f i r . Y, z ) = 1 p ( t , y, 2n + z ) + 1 dt, y , 2n - z ) n = - Q w = - w

2 sup f i t , y , z ) - inf fit, y, z ) I - J27;;' O S X ~ L O S Z S I

> 0. In particular, p is Lebesgue measure on [O, I],

1; G(x, w ) dw = 0 for all x E Rd by assumption.

3. DRIVING PROCESS IN A NONCOMPACT STATE SPACE

Let Y be an Ornstein-Uhlenbeck process, that is, a difiusion process in 118 with genera tor

(3.1)

where Y g ( y ) = g"(y) - yg'(y). The analogue of ( I . 18) is B~ = ((g, Y g ) : g E QR) c'(w, gg E @w},

Yc(s) ds, Yc(r) = Y ( ; ) .

and one might ask whether the analogous conclusion holds. Even if Theorem 2.4 could be extended to the case of E locally compact (with C(E) replaced by t ( ( E ) ) . i t would still be inadequate for at least three reasons. First, (2.22) is not satisfied. Second, convergence in (2.23) cannot be uniform if the right side is nonzero. Third, with G(x, y) = y, we have H ( x , y) = y , which is not even bounded in y E R. much less an element of 9(Bo). This last problem causes the

most difficulty. We may be able to find an operator Y that formally satisfies Bo Vg = -g (e.g., if Bo is given by a differential operator Y, we may be able to solve WI = -g for a large class of g), but Yg # 9 ( B o ) for the functions g in which we are interested.

There are several ways to proceed. One is to prove an analogue of Proposi- tion 2.2 with the role of C,(E) played by the space of (equivalence classes of) Bore1 measurable functions f: E-+ L with I I f ( . ) I l E L'(p), where p is the stationary distribution of Y. However, this approach seems to require that Y have initial distribution p.

Instead, we apply Corollary 8.7 (or 8.16) of Chapter 4, which was formulated with problems such as this in mind. The basic idea in the theorem is to "cut ofl" unbounded Vg by multiplying by a function 4' E C,(E), with 4c = 1 on a large compact set, selected so that 4 , V g E 49(Bo) and Bo(+ Yg) is approximately -g. We show, in the case of (3.2), that X' =S x + $W in C,[O, 00) as E+ O + .

3.1 Theorem Let E be a locally compact, separable metric space, let F. G E

C2*O(R' x E), and that cmdRd X E), and Suppose that Fl, ..., F d E: C'*O(w' X E), that G I , ..., Gd E

(3.3)

for every compact set K c E. Let {S(t)} be a Feller semigroup on c(€) with generator B, and let p E iP(E). Let p : E - , (0, 00) satisfy l/p E e ( E ) , let tj~ E

C,"[O, 00) satisfy xlo, s 4 z; xlo, 2,, fix 0 < 8 < 1, and define 4, E CJE) and K, c E by

(3.4) ~ c ( Y ) = 4('P(Y)) and K 8 = {Y E E : eeP(Y) S I}.

Assume that +& E d(Bo) for each E E (0, 1) and

(3.5) sup I Bo 4,(y) I = O ( E ~ ) as E-, 0. Y e Kt

Define

(3.6) = {g E C(W' x €1: J sup Idx, y)ip(tiy) < oo for 1 = 1,2, ... ,

and let V be a linear operator on & with 9 ( V ) c {g E A: g(x, y)cl(dy) = 0 for all x E R'} such that if g E 9 ( V ) , then (VgWx, .)tj~,(.) E 9(Bo) for every xER'andO<c< land

(3.7)

1x1 s J 1

SUP I BoC(VgHx. .)~.(.)I(Y) + AX, y)l = 41) 8s 8- 0 1x1 $1. Y Ks

for I = 1, 2, ... . Assume that /E C(Rd) and g E 9(Y) imply fg E 9(V) and W g ) = s Vg.

3. DRIVING m o c E s s IN A NONCOMPACT STATE SPACE 481

The following assumptions and definitions are made for i , j , k = I, . , . , d. Suppose G, E 9 ( V ) and the left side of (3.7) with g = G, is O(E) as E - 0 for I = I , 2, . . . . Assume H, = VG, E C2*o(Wd x E) and F , , G , H,, G , dH,/dx, E .A. Suppose a,, and b, , defined by (2.27) and (2.28). belong to C1(Rd). Suppose G,H, f G,H, - a,, E W V ) , F, + G . V, H, - b, E g ( V ) , ii,, 3 V ( G i I f , + G,H, - a,,) E C'*o(Wd x E), 6, 1 V(Fi + G . V , Hi - h,) E C1*o((wd x E).

Assume that H , , F, H , , F , a H i l a x i , G, i l k , G, a i , , l ax , , G , 6 , , G , ab , /dx , , when multiplied by the function ( x , y ) ---. xlo, ,,( I x [ ) / p ( y ) , are bounded on Wd x E for I = I, 2, . . . . Assume further that ii,,, 6,, F,c?,,, F, dii,,/dx,, F , 6 , , Fi ab',/ax,, when multiplied by the function (x, y ) -+ xl0. ,,( I x I ) /p(y) ' , are bounded on Rd x E f o r l = t,2, . . . .

Fix po E P(E), and let Y be a Markov process corresponding to {S( t ) } with sample paths in D,[O, 00) and initial distribution p0. Assume that

and

(3.8)

for each T > 0. Fix xo E Wd and define Xg for 0 -= E < 1 to be the solution of the differential equation (2.25) with initial condition X'(0) = xo. Put

Then C is dissipative. Assume that c, which is single-valued, generates a Feller semigroup { T(t)) on C(Rd), and let X be a Markov process corresponding to { T(I ) } with sample paths in C,,[O, 00) and initial distribution d,,,. Then X' * X in CR,[O, m) as E -+ 0.

3.2 Remark Instead of assuming an ergodicity condition such as (2.29), which would be rather diilicult to exploit here (and may be rather difficult to verify), we assume the existence of a linear operator V such that (essentially) Eo V = - I . 0

Proof. For each E E (0, I), exactly the same argument as used in the proof of Theorem 2.4 shows that { (Xr( t ) , Y(I/E')), t 2 0) is a progressive Markov process in Rd x E corresponding to a measurable contraction semigroup with full generator that extends { ( J A,/):/E 9}, where

(3.10) 9 = { J E C,!*"(Rd x E ) : f ( x , .) E 9(Bo) for all x E Rd}

and

(3.11) A t f ( ~ . Y) = { F ( x , Y ) + E - ' G ( x , y ) ) * V , J ( x $ y ) + E - ~ B O [ J ( X , . ) ] (y ) .

(Note that i f f € 9, the function ( x , y)-+ E o [ j ( x , . ) ] ( y ) is automatically jointly measurable.) Let (1. g) E C and define

(3.12) h , = V(G . Vf) = H ' Vf

482 RANDOMEVOLUTIONS

and (3.13) ha = V(F * Vf + G V,hl - g)

I d d

= - c 4,4 a,.f+ c 6, a i l : 2 i.j= 1 i= 1

LCX, Y ) = (f(x) + W X , Y) + E2h,(X. Y)W'(Y),

For each E E (0, l), definef, E 9 by

(3.14)

and observe that (3.15) A, f,k Y ) = 8 - ' f ( X P 0 4 k Y )

+ E - W X . Y ) * V/(X)4AY) + BoChAx, *W,(.)I(Y))

+ {W. Y) v m + (4x9 v) - V,h,(x, Y ) M d Y )

+ BoChdx, *)#d*)l(Y)

+ 4m. Y ) . V*h,(x, Y ) + a x , Y) . V,h2(x, Y))4r(Y)

+ &zm, Y) - v, h2(x, Y)4I(Y)

for all (x, JJ) E Rd x E. By (3.5) and the other assumptions,

(3.16) SUP SUP IL(X9 Y)l < 0%

(3.17) lim SUP I A ( X . Y ) -/(XI I = 4

(3.18) lim SUP I A, L k Y) - g(x) I = 0.

In view of (3.8), the result follows from Corollary 8.7 of Chapter 4. 0

3.3 Example Let E = R and define Bo by (3.I), where Yg(y) = ~ " ( y ) - yg'(y). I t is quite easy to show that the Feller semigroup {S(t)} on e(W) generated by 8, has a unique stationary distribution p, that p is N ( 0 , I), the standard normal distribution, and that

(3.19)

However, these results are not explicitly needed.

o < c < l (1. y ) c W n &

1-0 ( x . y ) e W ~ Y .

1-0 ( I , Y ~ ~ W R &

bp-lim If b ( s ) g ds - /g(z)p(dz)( = 0,

For each n 2 1, define 6. : W --+ (0, 00) by &(y) = (1 + Y')"'~,

g E e(R). 1 - 9 ,

g E C(Rd x R): SUP - ' ' x , ~ ) ' < o o for / = 1 , 2 , ... ~ 1 ~ s I , y o o a 4AY)

and dm = A,. Define V on

(3.21) 9(V) = g(x, y)e-y2'2 dy = 0 for all x E Rd

4. NON-MARWOVIAN DRIVING PROCESS 183

(3.22) Vg(x , y ) = [ez2(' lmg(x , w)e-w' '2 dw dz,

and note that Y: 9 ( V ) n dn+Am for each n L I . Also, if g E 9 ( V ) n C"O(Wd x W) and IV,gl E .Aw, then V g E C'*O(IWd x Iw) and, for i = 1, , . . , d, dgpx, E 24( V ) and a( Vg) /dx , = V(dg/sx,).

F i x m z 1 and let p = &,,, and 0 = f. Observe that V satisfies the required conditions (in fact (3.7) is zero). Assume, in addition to the assumptions on F and G in the first sentence of the theorem, that G, E g ( V ) n A,,, and F , , dF , /dx , , Gi, dG,/dx,, d2Gi/dx, ax, E ,rY, for i, j , k = 1, ..., d. If c satisfies the condition of the theorem, the only condition that remains to be verified is (3.8). For this it suffices to show that

2 3mlZ

(3.23) lim sup . I / ' ( 1 + Y(;) ) = 0 a.s., t 2 0. c - 0 Oassf

For the latter it is enough to show that for each 1. > 0 there exists a random variable q such that

To verify (3.24). we need only show that lim,,,, 1 Y(c)l /rA = 0 as . for every 1 > 0, which follows from the representation

(3.25) Y ( t ) = e"Y(0) -t e-'W(ez' - I ) ,

where W is a standard one-dimensional Brownian motion, and the law of the iterated logarithm for W . 0

4. NON-MARKOVIAN DRIVING PROCESS

We again consider the limit in distribution as c--+ O + of the solution X" of the differential equation

(4.11

driven by Y(./e2), where Y is a process in a compact state space. However, instead of assuming that Y is Markovian and ergodic as in Section 2, we require that Y be stationary and uniform mixing.

184 UNDCMAEVOLUnONS

4.1 Theorem Let E be a compact metric space, let F, G: IW' x E 4 Rd, and suppose that F , , ..., Fd E C'*'(Rd x E), G I , ..., G, E C2*'(Rd x E), and

(4.2)

Let Y be a stationary process with sample paths in DEIO, oo), and for each r 2 0, let 9, and 9' denote the completions of the a-algebras 9: and rr{ Y(s): s 2 t ) , respectively. Assume that the filtration {9J is right continuous, and that

(4.3)

satisfies

(4.4)

Suppose that

(4.5) E[G(x, Y(O))] = 0, x E W?

Fix xo E Rd, and define X' for 0 < E < 1 to be the solution of the differential equation (4.1) with initial condition XYO) = xo. Put

where

and

(4.8) b,(x) = E[Fi(x, Y(O))] + L E [ G ( x , Y(0)) V,Gl(x, Y(t))] dr.

Then C is dissipative. Assume that c, which is single-valued, generates a Feller semigroup { T(r)} on e(Rd), and let X be a Markov process corresponding to (T(t)) with sample paths in Cp(,[O, 00) and initial distribution I&,,. Then X' 4 X in Cw[O, 00) as E + 0.

Proof. Let t, u L 0 and let X be essentially bounded and S'+"-measurable. Then by Proposition 2.6 of Chapter 7 (r = 1, p = oo),

4. NON-MARKOVIAN DRIVING PROc(ESS 485

where p(u) is defined by (4.3). For example, conditioning on 4, and using (4.5). we find that

for all x E Rd, t 2 0, and i , j = I , . .., d. The same inequality holds when G, and/or G, are replaced by any of their first- or second-order partial x- derivatives, and therefore the coeficients (4.7) and (4.8) are continuously differentiable on Rd.

We also observe that the diffusion matrix (aJx)) is nonnegative definite for each x E Rd. For if x, 4 E Rd and T > 0,

+ GAx, Y(0))Gi(x, Y ( t - s))] ds dr

As T -4 m, (4.1 1). which is nonnegative, converges to

(4.12)

Thus, C satisfies the positive maximum principle, hence C is dissipative (Lemma 2. I of Chapter 4) and C is single-valued (Lemma 4.2 of Chapter I).

The growth condition (4.2) guarantees the global existence of the solution X K of (4.1). Denote by (bfj the filtration given by 9; = 45,,a2, and let ,& be the full generator of the associated semigroup of conditioned shifts (Chapter 2, Section 7). By Theorem 8.2 of Chapter 4, the finite-dimensional distributions

186 R A N D O M EVOLUTIONS

of X' will converge weakly to those of X if for each ( J Q) E C, we can find (r, g') E 2' for every E E (0, 1) such that

(4.14)

(4.15)

and

lim E[ I jYr) --f(X'(t))I 1 = 0, r z 0, r - 0

(4.16) lim EC I @(t) - g(X'(t)) I ] = 0, t L 0. 8 - 0

By Corollary 8.6 of Chapter 4 and Problem 25 of Chapter 3, we have XI X in C,,[O, 00) as E- 0 if (4.14) and (4.15) can be replaced by the stronger conditions

(4.17)

and

(4.18)

Ig'(t)J c 00, T > 0, 1 lim .[ sup I j'(r) -j(x'(i))l] = 0, T > 0. c-0 I C Q n (0 . TI

Fix (1 g) E C, and let E E (0, 1) be arbitrary. We let

where the correction terms fi , & E 9(&) are chosen by analogy with (3.12) and (3.13). Let us first consider f ; " l . We define f; : Rd x [O, 00) x Q-+ W by

(4.20) JW t , 4 = G ( x , Y (,'i, o)) - Vf(x).

Clearly,/C, is A?(R') x Afro, 00) x 9-measurable and is C: in x for fixed (I, 0).

In fact, there is a constant k , such that/;(x, f , o) = 0 for all 1x1 L k, f 2 0, and o E R, and

4. NON-MARKOVIAN DRIVING PROCESS 187

for all t 2 0 and o E Q, where ll/llcm = ~I,I,,IIDyll. By Corollary 4.5 of Chapter 2 there exists gt: Iwd x [O, 60) x [O , 00) x R 4 R, a ( R d ) x 9[0, 60)

x 0-measurable, Cf in x for fixed (s, t , w), such that

(4.22) g"lx, S, t, w) = E:[ /L , (x , t + S, . ) 1 ( 4

for all x E W d and s, I 2 0, where Ef denotes conditional expectation given 9: here and below. Moreover, g ; may be chosen so that g,(.u, s, 1, Q) = 0 for all I X I 2 k, , s, t 2 0, and UJ E R, and

(4.23)

for all s, f 2 0 and o E Q. The latter can be deduced from (424 (4.9), and (4.21). We now define h: : Iwd x [0, a) x Q--+ (w by

(4.24)

Clearly, / I : i s 9 ( R d ) x @?-measurable and i s Ct in x for fixed (I, cu). In fad, h",x, f, o) = 0 for all I x I 2 k, , t ;r 0, and w E R, and

(4.25)

for all t 2 0 and o E R. Finally, we define hc, : [0, ou) x R -+ R by

(4.26)

I t follows that h'i i s optional (hence progressive).

0. Clearly,

(4.27)

llg"t.* s, t , 4 l l c 2 2Y4J 2 (9 h " , ~ , r , 0) = E - ' q ; (x , s, t, 0)) ds. r

IIh;(', t, w)IIc, I ZY d ~ ) ds 1: & ( I , 0 ) = h",(X'(c, 0). I , 0).

To show that fi E 9(&, we apply Lemma 3.4 o f Chapter 4. Fix r , > I , 2

h W ( f 2 ) . t z ) - h;(X"l,) , [*I

4)

= & - 2 { [ g ; ( x t ( t , ) , s + t2 - t l , t , ) ds - &(X'(tl), s, t I ) .) = - e - 2

i a - 1 1

gt (x ' ( r , ) , s, I , ) ds

= - & - ~ € ; ' [ p ; ( X ~ ( f , ) , s) d.1.

Finally, we must verify condition (3.15) of Chapter 5, which amounts to showing that, for each t 2 0,

(4.30) lim E [ I V , h",(XYt), t + S ) - V x h;(Xe(t), t ) I ] = 0.

(We can ignore the factor F + E-'G because V,h;(x, t, w) has compact support in x, uniformly in (t, a).) Using the bound (4.23), the dominated convergence theorem reduces the problem to one of showing that, for each s, t 2 0,

(4.31) lim €[ lV,g , (Xc( t ) , s, t + S, .) - V,&(X'(t), s, t , .)I] = 0

or

(4.32)

But (4.32) follows easily from the right continuity of Y and the right continuity of the filtration (9;). We conclude from Lemma 3.4 of Chapter 4 that

1-0+

1-O+

lim E[ I E:+d[V, f t (X' ( t ) , t + S + s)] - Ef[Vi j f (X' (r ) , + s)] I ] = 0. d-O+

(4.33) ( f i ( t ) , {F(x, y ) + E - I G ( X , y)} * V,hf(x, t ) - E - ~ G ( x , y) . V!(X)) E d',

where x = X'(t) and y = Y(t /e2) . We turn now to the definition of &). We define f '1: RJ x [0, 00) x R-, W

by

(4.34) / ; ( x , t , O) = F ( X, Y ( ; , o ) ) * v f ( x ) + G ( x . .(;,o))

vx h'l(x. 1 . 0 ) - g(x).

Observe that/', is 4?(WJ) x U-measurable and is C,! in x for fixed ( t . 0). In fact, there is a constant k2 such that f2(x, t , o) = 0 for all I x I L k2, t 2 0, and w E n, and

4. NON-MARKOVIAN DRIVING PROCESS 183

by (4.25). We now define 9:. h", and fi by analogy with g; , h", and f;; . The only thing that needs to be checked is the analogue of (4.23), which is that, lor appropriate constants cI , c2 > 0,

for all s, f 2 0 and o E 0. Observe that the right side of (4.36) is Lebesgue integrable on [0, 00) by (4.4).

To justify (4.36), fix x E 88' and s, t 2 0. Then

1 - E [ C ( x . Y ( y ) ) . V x h : ( x , t + S ) 1

by the definition of C. Consequently, a similar equation holds for V,&(x, s, t , .) with each integrand replaced by its x-gradienl. By (4.9),

I; 2+) SUP I F(X, Y) ' V m l .

Since V x h;(x, t + s) = E - ~ @ V,g;(x , s', r + s) ds', we need lo consider


for fixed s' 2 0. By (4.9), this is bounded by

(4.40)

Moreover, conditioning on Sf+, and applying (4.9) and (44, each of the two expectations in (4.39) is bounded as. by

Thus, (4.39) is bounded by c3 rp((s V s')/E') for an appropriate constant c3 . Similar bounds hold when all integrands arc replaced by their x-gradients, and thus (4.36) can be assumed to hold. It follows from (4.36) that

(4.42)

for all t 2 0 and o E R.

word, to show that The argument used to show that 1;; E O(&) now applies, almost word-for-

where x = Xc( t ) and y = Y(t/E'). The only point that should be made is that, in proving the analogue of (4.32)- V7,fi(Xe(t), t + 6 + s) no longer converges pointwise in w, but only in LL. However, this suflices.

Clearly,

where x = Xc( t ) and y = Y(c/E'). Recalling (4.19), we obtain from (4.44), (4.33). and (4.43) that

where x = Xc((t) and y = Y(t/&'). By (4.25) and (4.42). together with the fact that V,&(x, t , w ) and V,h;(x, t, w ) have compact support in x, uniformly in (1, w), we see that (4. 13H4.18) are satisfied, and hence the proof is complete. 0

6. NOT€S 491

5.

1.

2.

3.

4.

6.

PROBLEMS

Formulate and prove a discrete-parameter analogue of Theorem 2.4. (Both X and Y are discrete-parameter processes, and the differential equation (2.25) is a difference equation.)

Give a simpler proof that X c * x + f i W in (3.2) by using the representation

(5.1) Xc(t) = x + EY(O) + &W(;> - EY(’) E 2 ’ t 2 0,

where W is a one-dimensional Brownian motion.

Generalize Example 3.3 to the case in which Y ( t ) E UZ(f), where

(5.2) dZ(t) = S dW(t) + NZ(t) dt

and U , S, and N are (constant) d x d matrices with the eigenvalues of N having negative real parts, and W is a &dimensional Brownian motion.

Extend Theorem 4.1 to noncompact E. The extension should include the case in which Y is a stationary Gaussian process.

NOTES

Random evolutions, introduced by Griego and Hersh (1969). are surveyed by Hersh (1974) and Pinsky (1974).

The derivation of the telegrapher’s equation in Section I is due to Goldstein (1951) and Kac ( 1 956).

The results of Section 2 were motivated by work of Pinsky (1968), Griego and Hersh (1971). Hersh and Papanicolaou (1972). and Kurtz (1973).

Theorem 4. I is due essentially to Kushner (1979) (see also Kushner (1984)). though the problem had earlier been treated by Stratonovich (1963, 1967), Khas’minskii (1966). Papanicolaou and Varadhan (l973), and Papanicolaou and Kohler ( I 974).

APPENDIXES

1. CONVERGENCE OF EXPECTATIONS

Recall that X , z X implies X,,A X implies X, =S A', so the following results, which are stated in terms of convergence in distribution, apply to the other types of convergence as well.

1.1 Proposition (Fatou's Lemma) Let X, 2 0, n = 1,2, . . . , and X , * X. Then

Proof. For M > O

- lim ECX,] 2 lim E [ X , A M] = E[XA MI, n-m (I- m

(1.2)

where the equality holds by definition. Letting M-+ 00 we have (1.1). 0

1.2 Theorem (Dominated Convergence Theorem) Suppose

IXnls Y., n = 1,2 ,..., X,P,X, Y , = Y

and limn4m ELK] = ECY]. Then

(1.3) lim E[Xn] = E [ X ] . n-m

492



1. UNIFORM INTECRA8ILITY 493

proof. It is not necessarily the case that (Xn, Y,) * ( X , Y). However, by Pro- position 2.4 of Chapter 3, every subsequence of { ( X , , V. ) ] has a further subsequence such that (XI,, <,)=s(x, p), where and X have the same distribution, as do p and Y. Consequently, Y,, + X n , - p + f and V,, - X,, =5 P - 2, so by Fatou's lemma,

- lim (EIYn,l + ECX,,,]) 2 ECYl + ECX] h-m

( 1.4)

and

- lim (€[<,I - E[Xn,]) 2 ECYI - ECXl. h-m

(1.5)

Therefore Iimk-- ECX,,] = ECX], and (1.3) follows. 0

2. UNIFORM INTEGRABILITY

A collection of real-valued random variables { X,} is uni/ormly integrable if sup, E[ I X, 13 < 00, and for every E > 0 there exists a b > 0 such that for every m, f (A , ) c 6 implies I E [ X , xA.] I < E.

2.1 Proposition The following are equivalent :

Proof. Since P { IX,I > N } s N - ' E [ ( X , ( 1, it is immediate that (a) implies (b). More precisely,

N P { I Xm I > Nl s E [ ~ l l x , l , N I I X a I 1 9

and since

491 APPENDIXES

2.2 Proposition A collection of real-valued random variables {X,) is uniformly integrable if and only if there exists an increasing convex function cp on LO, 00) such that limx,, cp(x)/x = a0 and sup, E[tp(IX,I)] < a0.

Proof. We can assume cp(0) = 0. Then cp(x)/x is increasing and

(2.3)

Therefore sufficiency follows from Proposition 2.l(b). By (b) there exists an increasing sequence ( N , } such that

Assume No = 0 and define q(0) = 0 and

2.3 Proposition If X, *X and {X,} is uniformly integrable, then limn-.m E[X,] = ECX]. Conversely, if the X, are integrable, X,*X, and

E l I X, I ] = E[ I X I], then {X,} is uniformly integrable.

Proof. If (X,) is uniformly integrable, the first term on the right of

(2.6) ECIXnll = ECIXnI - NAIXnIl+ ECNAlXnIl

can be made small uniformly in n. Consequently lirn#-,, EC I X , I] = E[ I X I I, and hence limn-,m E [ X , ] = ECX] by Theorem 1.2.

(2.7)

of Proposition 2.1 follows. 0

Conversely, since

lim E [ ) X , ) - NAIX,I] = E[IXl] - E[NAJXJ]

and the right side of (2.7) can be made arbitrarily small by taking N large, (b) n-m

2.4 Proposition Let (X,) be a uniformly integrable sequence of random variables defined on (n, .%, P). Then there exists a subsequence {X,,} and a random variable X such that

lim E[X,ZJ = E [ X a k - m

for every bounded random variable Z defined on (n, .F, P).

2.5 Remark The converse also holds. See Dunford and Schwartz (1957), page 294. 0

3. BOUNDED POINTWISE CONVERGENCE 495

Proof. Since we can consider {X,VO} and { X , A O ) separately, we may as well assume X, z 0. Let d be the algebra generated by { { X , < a ) : n = I , 2, . . . , a E 691. Note that .d is countable so there exists a subsequence { X,,J such that

(2.9)

exists for every A E d. Let Y be the collection of sets A E 9r for which the limit in (2.9) exists. Then d c (9. If A , B E Y and A c B, then B - A E Y. The uniform integrability implies that, if { A , ) c Y and A I c A2 c * * * , then u k Ak E 9. ( p ( u k A, - A,,,) can be made arbitrarily small by choosing m large.) Therefore the Dynkin class theorem (Appendix 4) implies 49 2 a(.d).

Clearly p is finitely additive on a(&), and the uniform integrability implies p is countably additive.

Clearly p << P on a(.d), so there exists a a(.d)-measurable random variable X such that p ( A ) = E [ X z A J , A E 44.

By (2.91,

(2.10) lim E [ X , , , Z J = E [ X Z J k - m

for all simple a(&)-measurable random variables and the uniform integrability allows the extension of that conclusion to all bounded, n(.d)-measurable random variables. Finally, for any bounded random variable Z,

(2.1 I) lim ECX,, 21 = lirn E [ X , , ECZ I u( .d ) ] ] k - m k - m

= E [ X E [ Z ) a ( . d ) J J = E [ X Z ] . 0

3. BOUNDED POINTWISE CONVERGENCE

Let E be a metric space and let V ( E ) denote the space of finite signed Bore1 measures on E with total variation norm

(3.1 )

A sequence If,} c B(E) converges in the weak* topology to J (denoted by w+-lim,,, = f ) if f dv for each v E V ( E ) . A sequence {I,} converges boundedly and pointwise to/(denoted by bp-lim,,, f, =f) if sup,lIf,Il < 00 and limn4a, j , ( x ) = f ( x ) for each x E E.

f j , dv =

3.1 and only if bp-limn,, J, =J:

Proposition Letf,, n = I , 2, ..., andJbelong to B(E). ~ + - l i m , , ~ f, = f i r

3.2 Remark This result holds only for sequences, not for nets. 0

4% APPENDIXES

Proof. If w+-lim,,,, f, =$. then sup,, 11 f, d v ( < 00 for each u E V ( E ) and the uniform boundedness theorem (see e.g., Rudin (1974), page 104) implies sup,, Ilf.ll < 00. Of course, taking v = 6, implies limn,, jJx) =/(x).

0 The converse follows by the dominated convergence theorem.

Let H c B(E). The bp-closure of H is the smallest subset R of B(E) containing H such that {J,} c I? and bp-lim,,,, = f imply f E A'. Note the bp- closure of H is not necessarily the same as the weak+ closure. For example. let E = [ O , 1Jand H = { n ~ ~ I I , ~ ~ , r r + , u , , l , : O ~ k < n ' , n = 1 , 2 , ...}. ThenHisbp- closed, but it is not closed in the weak+ topology.

4. MONOTONE CUSS THEOREMS

Let R be a set. A collection A of subsets of n is a monotone class if

(M 1) ( A , , } c A and A, c A 2 c . . . imply u A , , € &

and

(Ma ( A , , } c & and A , a A A l = * * - imply n A , E A .

A collection 9 of subsets of fl is a Dynkin class if

(DI) n € 9,

(D2) A, E c 9 and A c B imply B - A €9,

(D3) { A , , } c 9 and A l c A 2 c * . . imply u A , E ! ~ .

4.1 Theorem (Monotone Class Theorem) I f d is an algebra and A is a monotone class with d c A, then a(d) c A.

Proof. Let M(sd) be the smallest monotone class containing d. We want to show M(d) = o(d). Clearly it will be suliicient to show that M ( J ) is a 0-

algebra. First note that { A E M ( d ) : A' E M(J)} is a monotone class that contains

sd and hence M ( d ) , that is, A E M ( J ) implies AC E M ( d ) . Next note that for A E J, { B : A u B E M(A)] is a monotone class containing d and hence M ( d ) , that is, A E d and B E M ( d ) imply A u B E M(sd) . Finally, by this last observation, if A E M(d) then {B: A u B E Ad(&)} is a monotone class containing sd and hence M ( d ) , that is, M ( d ) is closed under finite unions. Since M ( d ) is closed under finite unions, by (M 1) it is closed under countable

0 unions, and hence is a a-algebra.

4. MONOTONE CLASS THEOREMS 497

4.2 Theorem (Dynkin Class Theorem) Let .Y be a collection of subsets of n such that A, E E Y implies A n E E 9‘. I f 9 is a Dynkin class with 9’ c 9, then 49) c 9.

Proof. Let D ( Y ) be the smallest Dynkin class that contains 9. It is sufficient to show that D(Y) is a a-algebra. This will follow from (D3) if‘ we show D ( 9 ) is closed under finite unions.

If A, E E 9, then A‘, ET, and A‘ u ZT = Cl - A n B are in D ( Y ) . Conse- quently A‘ n B‘ = A‘ u E - E, and A u E = R - A‘ n F are in D(Y).

For A E 9, { E : A u E E D(.Y’)} is a Dynkin class containing .Y, and hence A E Y and E E D ( 9 ) imply A u E E D(9’). Consequently, for A E D(.Y), { B : A u B E D ( Y ) } is a Dynkin class containing Y, and hence A, B E D(.Y)

n

AC u F - A‘ = A n ET, A‘ u E = R - A n IT,

implies A u B E D(9’).

4.3 Theorem Let H be a linear space of bounded functions on R that contains constants, and let Y be a collection of subsets of R such that A, B E .Y implies A n B E 9’. Suppose z A E H for all A E 9, and { , f n } c H, f , sfZ 5;

. . . , and sup. f , 15 c for some constant c implyf= bp-lim,,, J, E H. Then H contains all bounded a(,Y)-measurable functions.

Proof. Note that { A : x A E H} is a Dynkin class containing Y and hence a(Y). Since H is linear, H contains all simple a(.Y’)-measurable functions. Since any bounded a(.Y’)-measurable function is the pointwise limit of an increasing

0 sequence of simple functions, the theorem follows.

4.4 Corollary Let H be a linear space of bounded functions on R containing constants that is closed under uniform convergence and under bounded pointwise convergence of nondecreasing sequences (as in Theorem 4.3). Suppose H o c H is closed under multiplication ( J g E H , implies ,fq E H,). Then H contains all bounded a(H,)-measurable functions.

Proof. Let F E C(R). Then on any bounded interval, F is the uniform limit of polynomials, and hence , f ~ H , implies F(/) E H. In particular, Jm = [ l A ( f - u)VO]”” is in H. Note thatfl sJZ I . . . 5 I,and hence

(4.1) = lim S. E H . n - m

Similarly, forf,, . . ., f, E If,,

(4.2) X l l l . O l r .... I n ’ O r I E H

and, since o(H,) = a({ {fl > a , , . . . , f , > a,} : J , E H,, a, E R}), the corollary follows. 0

We give an additional application of Theorem 4.3.

4.5 Proposition Let E , and E2 be separable metric spaces. Let X be an E,-valued random variable defined on (n, 9, P), and let H be a sub-o- algebra of 9. Then for each JI E WEl x E2) there is a bounded B(E,) x #-measurable function cp such that

(4.3) ECW, Y) I J f I ( 4 = Cp(Y(4, a),

for every #-measurable, €,-valued random variable Y. If X is independent of M, then q does not depend on w. Specifically,

(4.4) rp(Y, 4 = Cp(Y) - E C W , YII .

Proof. If $(x, y) = g(x)h(y), then My, a ) = h(y)E[g(X)IJto]. Let H be the collection of t,b E B(El x E,) for which the conclusion of the proposition is valid, and let Y = { A x 8: A E a(El), B E a@,)}. Since x,, Ax, y ) = x,,(x)x,,(y), the proposition follows by Theorem 4.3. 0

5. CRONWALL'S INEQUALITY

5.1 Theorem Let y be a Borel measure on [O, a), let E 2 0, and let f be a Borel measurable function that is bounded on bounded intervals and satisfies

(5.1)

Then

(5.2) f ( t ) s ee"O1 '1, t 2 0.

In particular, if M > 0 and

(5.3)

then

(5.4) f ( I ) s ceM', I 2 0.

0 Sf(4 s E + I f(s)p(ds), t 2 0. lo. t)

0 Sf([) s 8 + M f ( s ) ds, I 2 0, l

6. THE WHlTNEY EXTENSION THEOREM 499

6. THE WHlTNEY EXTENSION THEOREM

For x E Rd and a E Hd, , let x m = n:= x;L, I a I = If-, ak , and a ! = nf= I ak!. Similarly, if D, denotes difTerentiation in the kth variable,

and iff is r times continuously differentiable on a convex, open set in Rd, then by Taylor's theorem

- D"'b/(x)J du (y - x)".

6.1 { f a : a E Zd, , I or1 s r } satisfiesja: E - , R for each a,

Theorem Let E c Rd be closed. Suppose a collection of functions

Then there existsje Cr(Rd) such thatf JE =lo

6.2 Remark Essentially the theorem states that a functionj, that is r times continuously differentiable on E can be extended to a function that is r times continuously differentiable on Rd. 0

Proof. The theorem is due to Whitney (1934). For a more recent exposition ' 0 see Abraham and Robbin (1967), Appendix A.

6.3 Corollary Let E be convex, and suppose E is the closure of its interior E". Supposef, is r times continuously differentiable on ED and that the derivatives D"/b are uniformly continuous on E". Then there existsje Cr(Rd) such t h a t j JEe =Jo.

Proof. Let Ra(x, y ) be the remainder (second) term on the right of (6.2). There exists a constant C such that for x, y E E",

(6.5) IRa(x, ~ ) l s Clx - y l ' - ' " ' ~ ( l ~ - .~l),

where

By continuity (6.5) extends to all x, y E E. Since Iim4-,, 4 6 ) = 0, the corollary follows. C I

7. APPROXIMATION BY POLYNOMIALS

In a variety of contexts it is useful to approximate a functionje C(Rd) by polynomials in such a way that not onlyfbut all of its derivatives of order less than or equal to r are approximated uniformly on compact sets. To obtain such approximations, one need only construct a sequence of polynomials (p,} that are approximate delta functions in the sense that for everyfe CJBP‘),

(7.1)

and the convergence is uniform for x in compact sets. Such a sequence can be constructed in a variety of ways. A simple example is

(7.2)

To see that this sequence has the desired property, first note that

(7.3)

For x in a fixed compact set and n sufficiently large

The second equality follows from the fact thatlhas compact support, and (7.1) follows by the dominated convergence theorem.

7. APPROXIMATION BY POLYNOMIALS 501

7.1 Proposition exists a polynomial p such that for every I a I I r,

Let f E C'(Wd). Then for each compact set K and E: > 0, there

x c K

Proof. Without loss of generality we can (Replacefby C . J where (' E CF(Rd) and C =

(7.6) pn(x) = [ , - (Y )Pn(x - Y) dy =

and note

assume f has compact support. 1 on K . ) Take

As an application of the previous result we have the following.

7.2 Proposition Let cp be convex on Rd. Then for each compact, convex set K and c: > 0, there exists a polynomial p such that p is convex on K and

(7.8)

Proof. Let p E CF(R') be nonnegative and

(7.9)

Then for n sufliciently large,

(7.10)

is infinitely differentiable, convex, and satisfies

E (7.1 I )

For 6 suniciently small, v2(x) E cp,(x) + b I x l 2 satisfies

SUP I&4 - cp,(x)l s? 3 ' x r K

(7.12)

502 m D l X p s

Recall that a function 4 E Cz(Rd) is convex on K if and only if the Hessian matrix ((Di D,$)) is nonnegative definite. Note that ((DI D,cpz)) is positive definite. In particular

(7.13)

By Proposition 7.1 there exists a polynomial p such that

(7.14)

and D, Dj p approximates D, D, cpz closely enough so that

(7.15)

Consequently, p is convex on K, and (7.12) and (7.14) imply p satisfies (7.8). 0

8. BIMEASURES AND TRANSITION FUNCTIONS

Let (M, A) be a measurable space and (E, r ) a complete, separable metric space. A function vo(A, E ) defined for A E A and B E 1 ( E ) is a birneusure if for each A E A, w,(A, a ) is a measure on a(€) and for each B E A?(& w o ( . , B) is a measure on 4.

8.1 Theorem Let vo be a bimeasure on +# x A?(€) such that 0 < vo(M, E) < co, and define p = yo(., E). Then there exists q : M x O(E)-r [O, 00) such that for each x E M, q(x, a ) is a measure on At(E), for each B E W(E), q ( . , E ) is A-measurable, and

(8.1) q(x, E)p(dx), A E A, B E O(E).

Furthermore,

defines a measure on the product a-algebra A x a(E) satisfying w(A x E ) = v,(A, E ) for all A E A’, B E @(E).

8.2 Remark The first part of the theorem is essentially just the existence of a regular conditional distribution. The observation that a bimcasure (as defined by Kingman (1967)) determines a measure on the product a-algebra is due to Morando (1969). page 224.

0. #MEASUIESAM)TRANUTIONRJNClWM 503

Proof. Without loss of generality, we can assume v,(M, E) = 1 (otherwise replace vo(A, 8) by v,(A, B)/v,(M, E)). Let { x , } be a countable dense subset of E, and let E l , B2 , . . . be an ordering of {B(x , , k - I ) : i = 1, 2, . . . , k = I , 2, . . .}.

For each B E @(E) , v 0 ( . , E ) c p, so there exists q O ( e r E), A-measurable, such that

(8.3) vo(A, B) = L V O ( X , B)lr(W, A E A.

We can always assume qo(x, E ) I; I, and for fixed E, C, with B c C, we can assume qo(x, B) $ qo(x, C) for all x. Therefore we may define qo(x, E) = 1, selwt qo(x, B,) satisfying (8.3) (with B = El), and define qo(x, 4) = 1 - qo(x, El), which satisfies (8.3) with B = SC, . For any sequence C , , Cz, ... where C, is B, or I$, working recursively we can select qo(x, C, n C, n * * . n C, n B,, ,) satisfying (8.3) with B = C1 n C, n . * n C, n Bk + , and q o ( x , C , n Cz n ... n C, n Bk+,)sqo(x ,Cl n Cz n * - . n Ck),anddefine

qo(x, CI n C, n n C, n B;+ ,) = qo(x, C, n Cz n * - * n C,)

-qO(x* CI cz n * ' * n c k n &+I),

which satisfies (8.3) with E = C , n C, n . . . n C, n S;+ , . For B E 9, = a(B,, . . . , B,), define qo(x, B) = qo(x, C, n Cz n - * n C,) where the sum is over {C, n C, n n C,: C, is B, or q, C, n C, n . - - n C, c E}. Then qo(x, E) satisfies (8.3) and qo(x, .) is finitely additive on urn 9,.

Let r, = {C, n C, n . a - n C,: C, is E, or &}, and for C e r, such that C # 0. let tC E C. Define q,(x, .) E 9(E) by

Note that for E E 9,,, q,(x, E ) = qo(x. E). For m = I , 2, .. . let K, be compact and satisfy vo(M, K,) 2 1 - 2-". For each m, there exists N, such that for n 2 N, there is a B E 9, satisfying K, c B t K,!,'". Hence

(8.5) I inf q,(x, KA/")y(dx) 2 qo(x, B)p(dx) 2 v,(M, K,) 2 I - 2-".

Therefore r z N , s

p x : inf q,(x, K:/") < I - nt- ' s m2-"', { szNm I (8.6)

and hence by Borel-Cantelli

(8.7) G = {x: 5 q,,(x, K:'"') 2 1 - m - ' for all but finitely many m

satisfies p(G) = 1. It follows that for each x E G, (qn(x. a ) } is relatively compact. Since limn-.* q,(x, B) = qo(x. B ) for every B E u. 9,,, for x E G there exists

I r-.m

!a4 A?rEra)[eF

q(x, .) such that q,(x, .) 30 ~ ( x , .). (See Problem 27 in Chapter 3.) By Theorem 3.1 of Chapter 3

a- m (8.8)

for all B E A?@) such that

Since for B E u, 9,

(8.10)

it follows from Problem 27 of Chapter 3 that (8.1) holds. 0

9. TULCU'S THEOREM

9.1 Theorem Let (Qk, s k ) , k = 1, 2, .. ., be measurable spaces, fl = R, x n2 x . Let PI be a probability measure on 9,.

and for k = 2, 3, ... let p k : n, x x n k , , x Pk-, 10, 1) be such that for each (0,. ..., 0,- ,) E a, x * * * x R,- I , P,(w,, ..., a,- , , .) is a probability measure on 9,, and for each A E 9 k , p k ( ' , A) is 9, x x 9k-I-measurable. Then there is a probability measure P on 9 such that

and 9 = 9, x P2 x

for A E 9 1 X ' * ' X F k ,

Proof. The collection of sets

d = { A X nk+, x R&+z X * * - : A E 9, X * * * X f i k , k = 1,2, ...}

is an algebra. Clearly P defined by (9.1) is finitely additive on d. To apply the Caratheodory extension theorem to extend P to a measure on a(&) = 9, we must show that P is countably additive on d. (See Billingsley (1979). Theorem 3.1 .) To verify countable additivity it is enough to show that {B,} c d, B1 3

B, =) ... and limn-- f(8,J > 0 imply 0, B, # 0. Let B,, = A, x &.+I

10. MEASURABLE SELECTIONS AND MEASU(LABlLl1Y OF INVERSES 505

(9.4)

Furthermore note that f k , . ;r f k , n + l so gk = bp-lim,,, I k , , exists, and by the monotone convergence theorem,

(9.5)

10. MEASURABLE SELECTIONS A N D MEASURABILIW OF INVERSES

Let (M. d) be a measurable space and (S, p ) a complete, separable metric space. Suppose for each x E M, rx c S. A measurable selection of (rx) is an A-measurable function8 M -+ S such that/(x) E I-x for every x E M.

10.1 Theorem Suppose for each x E M, r, is a closed subset of S and that for every open set U c S, {x E M : rx n U # 0) E A. Then there exist f,,: M -+ S, n = 1, 2, . . . , such that S. is &-rneasurable,S,(x) E r, for every x E M, and rx is the closure of {/I(x),j”(x), . . .}.

5 0 6 -

10.2 Remark Regarding x--, r, as a set-valued function, if (x E M: r, n U # 0) E -4 for every open U, the function is said to be weakly measurable. The function is measurable if "open" can be replaced by "closed." The theorem not only gives the existence of a measurable selection, but also shows that any closed-set-valued, weakly measurable function has the representation (known as the Castaing representation) r, = closure {j'&t),j1(x), . . .} for some countable collection of A-measurable functions. 0

Proof. See Himmelberg (1975). Theorem 5.6. Earlier versions of the result are 0 in Castaing (1967) and Kuratowski and Ryll-Nardzcwski (1965).

10.3 Corollary Suppose (M, A) = (E, for a metric space E. If ym E r,", n = I, 2, ..., and Iimm-- x, = x imply that {y,,} has a limit point in T,, then there is a measurable selection of {r,). 10.4 Remark The assumptions of the corollary imply that for K c E compact, Uz a r, is compact.

Proof. Note that for a closed set F. {x: r, n F # 0) is closed, hence measurable. If U is open, then U = Un F,, for some sequence of closed sets {F,,},

0 and hence {x: r, n U = 0) = u,, {x: r, n F,, = a} is measurable.

For a review of results on measurable selections, see Wagner (1977). One source of set-valued functions is the inverse mapping of a given func-

tion cp: E, -t E l , that is, for x E E , take r, = cp-'(x) = { y E El: cp(y) = x}. If cp is one-to-one, then the existence of a measurable selection is precisely the measurability of the inverse function. The following theorem of Kuratowski gives conditions for this measurability.

10.5 Theorem Let (S, , p l ) and (S2, pa) be complete, separable metric spaces. Let El E a@,), and let 9: El - S1 be Borel measurable and one-to- one. Then E2 = rp(E,) = (rp(x): x E El} is a Borel subset of Sl and cp-' is a Borel measurable function from El onto E l .

Proof. See Theorem 3.9 and Corollary 3.3 of Chapter I of Parthasarathy (1 967). 0

11. ANALMIC SETS

Let N denote the set of positive integers and X = N". We give N the discrete topology and X the corresponding product topology. Let (S, p) be a complete, separable metric space. A subset A c S is analytic if there exists a continuous function cp mapping JV onto A.

11. ANALVnC S€lS !%7

11.1 Proposition Every Borel subset of a complete, separable metric space is analytic.

Proof. See Theorem 2.5 of Parthasarathy (1967). 0

Analytic sets arise most naturally as images of Borel sets.

11.2 Proposition Let (S,, p , ) and (S2, p 2 ) be complete, separable metric spaces and let cp: S,-,S2 be Borel measurable. lf A EA?(S,) . then cp(A) = {cp(x): x E A } is an analytic subset of S2.

Proof. See Theorem 3.4 of Parthasarathy (1967). 0

11.3 Theorem Let (S, p) be a complete, separable metric space and let (a, 9, P) be a complete probability space. If Y is an S-valued random variable defined on (a 9, P) and A is an analytic subset of S, then { Y E A } E 9F.

Proof. See Dellacherie and Meyer (1978). page 58. The definition of analytic set used there is more general than that given above. The role of the paved set (F, 9) in the definition in Dellacherie and Meyer (page 41) is taken by (S, ii?(S)), and the auxiliary compact space E is where NA is the one- point compactification of N. Let B c E x S be given by B = ((x, cp(x)): x E N"}, where cp is continuous on N". Then for (z , } dense in

n - ' ) , where cl denotes the closure in (N')". Consequently B E ( X ( E ) x ii?(S)),,, ( ;Y(E) is the class of compact subsets of E) and A is the projection onto S of B, so A is A?(S)-analytic in the terminology of Dellacherie and Meycr. 0

S, B = 0. 0, Urn C I ( X E N": l x j l s m , j = 1, ..., n, CP(X) E B(z,, t t - l ) } x B(z,,

REFERENCES

Abraham, Ralph and Robbin. Joel (1967). Transwrsal Mappings and Flows. kqjamin. New York. Aldous, David (1978). Stopping times and tightness. Ann. Probab. 6,335-340.

Alexandroff, A. D. (1940-1943). Additive set functions in abstract spaces. Mat. Sb. 8, 307-348; 9.

Allain, Marie-France (1976). fitude d t la vitesse de convergence d’une suite de processus de Markov de saut pur. C. R. Acad. Scf. Paris 282,1015-1018.

Aim, Sven Erick (1978). On the rate of convergence in ditTusion approximation of jump Markov processes. Technical Report. Department of Mathematics, Uppsala University, Sweden.

Anderson, Robert F. (1976). Dimusion with second order boundary conditions, 1 , I I . Indiana Uniu. Math. J . 25,367-395.403441.

Artstein, Zvi (1983). Distributions of random sets and random selections. lsrael J . Math. 46.

Athreya, Krishna 8. and Ney, Peter E. (1972). Branching Processrs. Springer-Verlag, Berlin. Barbour, Andrew D. (1974). On a functional central limit theorem for Markov population pro-

cesses. Adv. Appl. Probab. 6.21-39.

Barbour, Andrew D. (1976). Sccond order limit theorems for the Markov branching process in random environments. Sfochasflc Process. Appl. 433-40.

Barbour. Andrew D. (1980). Equilibrium distributions for Markov population processes. Adu. Appl. Probab. 12,591614.

BBrlfai. PA1 (1966). Die Bostimmung dcs zu dnem wiederkehrenden Prozess gehorenden Verteil- ungsfunktion aus den mit Fehlern behafken Daten eiher Einziger Relation. Stud. Sci. Math.

563-628; 13,169-238.

3 13-324.

Hung. I. 161-168.

508



REFERENCES 509

BeneJ, VAclav Edvard (1968). Finite regular invariant measures for Feller processes. J . Appl. Probah. 5,203-209.

Bhattacharya. Rabi N. (1982). On the functional central limit theorem and the law of the iterated logarithm for Markov processes. Z. Wahrsch. verw. Gehiete 60. 185-201.

Billingsley. Patrick (1968). Convergence ojProbability Measures. Wiley. New York.

Billingsley. Patrick (1979). Prohabiliry and Measure. Wiley. New York.

Blackwell, David and Dubins, Lester E. (1983). An extension of Skorohod’s almost sure represent-

Blankenship. Gilmer L. and Papanicolaou, George C. (1978). Stability and control or stochastic

Borovkov. A. A. (1970). Theorems on the convergence to Markov ditrusion processes. Z. Wahrsch.

Breiman. Leo (1968). Probability. Addison-Wesley, Reading. Mass.

Brown, Bruce M. (1971). Martingale central limit theorems. Atin. Math. Sratist. 42,59-66. Brown, Timothy C. (1978). A martingale approach to the Poisson convergence of simple point

Castaing, Charles (1967). Sur les multi-applications mesurables. Rev. Francaise /nJ Rech. Opkra. 1.

Chenfov, N. N. (1956). Weak convergence of stochastic processes whose trajectories have no discontinuities of the second kind and the heuristic approach to the Kolmogorov-Smirnov tests. Theory Probab. Appl. 1. 140-149.

ation theorem. Proc. Amer. Math. Soc. 09,691-692.

systems with wide band noise disturbances 1. S I A M J . Appl. Marh. 34,437476.

uerw. Gebiete 16.47-76.

processes. Ann. Probab. 6,615628.

91-126.

ChernotT, Herman (1956). Large sample theory: parametric case. Ann. Math. Statist. 27, 1-22.

ChernoK, Paul R. (1968). Note on product formulas lor operator semigroups. J . Funct. Anal. 2.

Chow, Yuah Shih (1960). Martingales in a a-finite measure space indexed by directed sets. Trans.

Chung, Kai Lai and Williams, Ruth J. (1983). Inrroduction ro Srochusric Inrtgrurion. Birkhauser,

Cohn, Donald L. (1980). Measure Theory. Birkhauser. Boston. Coatantini, Cristina. Gerardi, Anna and Nappo, Giovanna (1982). On the convergence of

Costantini, Cristina and Nappo, Giovanna (1982). Some results on weak convergence of jump

Courdgc, Philippe (1963). IntCgrales stochastiques et martingales de c a d integrable. Stminaire

Cox. E. Theodore, and Rosler. Uwe (1982). A duality relation for entrance and exit laws for

Crandall. Michael G . and Liggctt, Thomas M. (1971). Generation or semi-groups of nonlinear

Csorgo. Mik16s and Rtvtsz. PHI (1981). Strong Approximations in Probability and Staristlcs. Aca-

Darden, Thomas and Kurtz, Thomas G. (1986). Nearly deterministic Markov processes near a

Davies, Edward Brian ( 1980). One-Parameter Semigroups. Academic, London.

Davydov. Yu. A. (1968). Convergence of distributions generated by stationary stochastic processes.

Dawson. Donald A. (1975). Stochastic evolution equations and related measure processes. J .

238-242.

Amer. Math. Soc. 97.254-285.

Boston.

sequences of stationary jump Markov processes. Statist. Probah. Lett. I . 155- 160.

Markov processes and their stability properties. Systems Conrrol Lerr. 2. 175- 183.

Brelot-Choquet-Deny. 7th year.

Markov processes. stochastic Process. Appl . 16. 141-151.

transformations on general Banach spaces. Anwr. J . Math. 93,265-298.

demic, New York.

stable point. To appear.

Theory Probab. Appf. IS, 691-696.

Mtrlriuar. Anal. 5. 1-52.

510 REmENcm

Dawson, Donald A. (1977). The critical m u s u n diffwion procar. 2. Wahrsch. wrw. Ge6iete 40, 125-145.

Dawson, Donald A. (1979). Stochastic measure dlffuaion proauer C a d . Math. Bull. 22, 129- 138.

Dawmn. Donald A. and Hochberg, Kenneth J. (1979). The a r r y i n ~ dimension of a measure diffusion proces. AM. Prabub. 7,693-703.

Dawson. Donald A. and Kurtz, Thomas 0. (1982). Applications of duality to measurevaluod processes. Advances in Filtering and Optimal Stocha.uk Control. &I. Notes Cont. IrJ: Sci. 42, Springer-Vcrlag, Berlin, pp. 177-191.

Dellacherie, Claude and Meycr, Paul-AndrC (1978). ProWflltles and Poteruld. North-Holland, Amsterdam.

Dellachcrie, Claude and Meyer, Paul-AndrC (1982). Probobilltin Md Potential 8. North-Holland, Amsterdam.

Dolcans-Dadc. Catherine (1969). Variation quadratique d a marcinpkr continua P droite. Ann. Math. Statist. 4Q. 284-289.

Doleans-Dade, Catherine and Meycr, Paul-And* (1970). IntCgnlcn rtochastiques par rapport aux martingales locales. S h i m i r e de Probabilltb I V , k t . Nates Math., 124, Springer-Veda& Berlin.

Donsker. Monroe D. (1951). An invariance principle for certain probability limit theorems. Man. A m . Math. Soc. 6.

Doob. Joseph L. (1953). Stochastic Processe~. Wiley, N m York. Dudky, Richard M. (1968). Distanas of probability mcuum and random varlabla. Ann. Moth.

Dunford, Nelson and Schwsrtz, Jacob T. (1957). Linrar Operafors Part I: General Theory. Wiley-

Dvorettky. Aryeh (1972). Asymptotic normality for sums of dependent random variables. Prac.

Dynkin, E. 8. (1961). Theory of Mar&ov Procases. PrenticcHdI, Er@cwood Cliffs, N.J. Dynkin. E. B. (1965). Mar&ou Processes I, 11. Springar-Verla& Berlin. Echeverria, Pedro E. (1982). A criterion for iovariant m u r u r a of Markov procwm. 2. Wahrwh.

Elliot, Robert J. (1982). Stocktic Calculus and Applicaflons. Sprhger-Verlag, New York. Ethier, Stewart N. (1976). A class of depera te diffusion proccsaca occurring in population

Ethicr. Stewart N. (1978). Differentiability preserving propertica of Markov semiPoups associated

Ethier. Stewart N. (1979). L h i t theorems for absorption times of genetic models. Ann. Probob. 7.

Ethicr, Stewart N. (1981). A clam or infinitedimensional diHuuons occumng in population

Ethicr, Stewart N. and Kurtz, Thomas 0. (1981). The inlnitely-many-neutral-allela diffusion

Ethicr, Stewart N. and Kurtz, Thomas G. (1986). The inRnitely-nuny-alleles model with selection

Ethier. Stewart N. and NaOylaki Thomar (1980). Diffusion approaimation of Markov chains with

Ewens, Warren J. (1972). The sampling theory of selectively neutral alleles. Theor. Pop Biol. 3,

Ewens, Warren J. (1979). Mathematical Population Genetics. Springer-Verlag, Berlin.

Statist. 39,15634572

Interscience. New York.

Sixth Berkeley Symp. Math. Statist. Prob. 2, University of California Press, pp. 513-535.

verw. &Mete 61. 1-16

genetics. Camm. Pure Appl. Math. 29,483-493.

with one-dimensional diffusions. 2. Wahrsch. mrw. Ge6iete U, 225-238.

622-738.

genetics. Indiana Univ. Moth. J . 30,925-935.

model. Ado. Appl. Probob. 13.429-452.

as a measure-valued diffusion. To apprar.

two time sales and applications to population genetics. Adu. Appl. Prohb. 12. 1449.

87-1 12.

REWIIWCES 511

Feller, William (1951). Diffusion processes in genetics. Proc. Second Berkeley Symp. Math. Statist.

Feller. William (1952). The parabolic di!krential equations and the associated semi-group of

Feller, William (1953). On the generation of unbounded semi-groups of bounded linear operatore.

Feller, William (1971). An lntroduction to Probability Theory and Its Applicafions / I , 2nd ed.,

Fiormza. Renato (19S9). Sui problemi di derivata obligua per le equadoni ellittiche. Ric. Mat. 8.

Fisher. R. A. (1922). On the dominana ratio. Proc. Roy. Soc. Edin. 42,321431.

Fleming. Wendall H. and Viol, Michel (1979). Some measurr-valued Markov proasses in popu-

Freidlin. M. 1. (1968). On the factorization of non-negative definite matrices. Theory Probab. Appl.

Friedman. Avner (1975). Stockostlc Diflerential Equations I . Academic. New York.

GYnsler. Peter and HYusler. Erich (1979). Remarks on the functional central limit theorem for

Gihman. 1. 1. and Skorohod. A. V. (1969). Infroduction to the Theory of Random Processes. W. 8.

Gihman, 1. 1. and Skorohod. A. V. (1972). Stochastic Diflerentiol Equations. Springer-Verlag.

Gihman, 1. 1. and Skorohod, A. V. (1974). The Theory of Stochastic Processes 1. Springer-Verlag,

Goldstein, Jerome A. (1976). Semigrouptheoretic proofs of the antral limit theorem and other

Goldstein, Sydney (1951). On difhsion by discontinuous movement& and on the telegraph equr-

Gray, Lawrena and Grifkath. David (1977.). On the uniqueness and nonuniquenas of proximity

Gray, Lawrence and Grifkath, David (l977b). Unpublished manuscript.

Griego, Richard J. and Henh, Reuben (1969). Random evolutions, Markov chains, and systems of

Griego. Richard J. and Henh. Reuben (1971). Theory of random evolutions with applications to

Grigclionis, Bronius and MikukviEius. R. (1981). On the weak convergence of random point

Grimvall. Anden (1974). On the convergence of sequen#s of branching proasses. Ann. ProCb. 2.

Guess, Harry A. (1973). On the weak convergence of Wright-Fisher models. Sfochartlc Process.

Gustofson, Karl (1966). A pcrturbation lemma. Bull. A m . Math. Soe. 72.334338.

Hall. P. (1935). On representatives of subsets. J . London Math. Soc. 10.2630. Hall, Peter and Heyde, C. C. (1980). Martingale Limit Theory a d Its Applicatlons. Academic. New

Hardin. Clyde (1985). A spurious Brownian motion. Proc. A m . Morh. Soc. 93.3SO. Harris, Theodore E. (1976). On a class or set-valud Markov proasm. Ann. Probab. 4, 175-194.

Prob., University of California Press. Berkeley, pp. 227-246.

transformations. Ann. Math. SS. 468-519.

Ann. Math. 58.166174.

Wiley. New York.

83-1 10.

lation genetia theory. lndiana Uniu. Math. 1. 28,817-843.

13.354-356.

martingaks. 2. Wahrsch. verw. Gebiete SO, 237-243.

Sounders Co., Philadelphia.

Berlin.

kr l in.

theorems of analysis. Semigroup Forum 12, 189-206.

tion. Quart. 1. M u h . Appl. Mafh. 4, 129-lJ6.

protxmcs. Ann. Probob. 5,678-692.

partial diLrentia1 equations. Proc. Not. Acod. Scl. USA 62.305-308.

partial diITercntia1 equations. Trans. Amer. Mafh. Soc. 156.405-418.

proasses. Lithwnian Math. Trans. 21.4945.

1027- 1045.

Appl. I. 287-306.

York.

512 IEFRENces

Hasegawa, Minoru (1961). A note on the conwrgence of suni-groups of operators. Proc. Japan

Helland, lnge S. (1978). Continuity of a class of random time transformations. Stochastic Process.

Helland, lnge S. (1981). Minimal conditions for weak convergence to a diffusion process on the

Helland. Ingc S. (1982). Central limit theorems for martinplea with discrete or continuous time.

Helms, Later L. (1974). Ergodic properties of several interacting Poisson particles. Adu. Math. 12,

Hersh. Reuben (1974). Random evolutions: a survey of rcsuIts and problems. Rocky MI. J. Math.

Hersh, Reuben and Papanicalaou, George C. (1972). Non-commuting random evolutions, and an

Heyde, C. C. (1974) On the central limit theorem for stationary processes. 2. Wahrsch. uerw.

Hille, Einar (1948). Functional Analysis and Semi-groups, Am. Math. Soc. Colloq. Publ. 31, New

Hille, Einar and Phillips, Ralph (1957). Funcrional Analysis and Semi-groups, rev. ed., Amer. Math.

Himmelbcrg, C J. (1975). Measurable relations. Fund. Math. 87,53-72. Holley, Richard A. and Liggett. Thomas M. (1975). Ergodic theorems for weakly interacting

systems and the voter model. Ann. Probab. 3,643-663. Holley, Richard A. and Stroock, Daniel W. (1976). A martingale approach to infinite systems of

interacting processes. Ann. Probab. 4, 195-228.

Holley, Richard A. and Stroock, Daniel W. (1978). Oeneralizsd Ornstcin-Uhlenbcck processes and infinite particle branching Brownian motion& Publ. RIMS, Kyoto Uniu. 14,741-788.

Holley, Richard A. and Stroock, Daniel W. (1979). Dual processes and their applications to infinite interacting systems. Adv. Math. 32,149-174.

Holley, Richard A., Stroock. Daniel W., and Williams, David (1977). Applications of dual processes to diffusion theory. Proc. Symp. Pure Math. 31. AMS, Providence. R.I., pp. 23-36.

Ibragimov, 1. A. (1959). Some limit theorems for strict-sense stationary stochastic processes. Dokl. Akad. Nauk SSSR, 115.71 1-714.

Ibragimov. 1. A. (1962) Some limit theorems for stationary prowsses. Theory Probab. Appl. 7, 349-302.

Ibragimov, 1. A. and Linnik, Yu. V. (1971). Indepndent and Statlonary Sequences of Random Variables. Wolters-Noordhoff. Groningen.

Ikeda, Nobuyuki, Nagasawa, Mamo, and Watmabc. Shinzo (1968, 1969). Branching Markov processes I, I1,lII.J. Math. Kyoto 8,233-278.365-410; 9.95-160.

Ikda, Nobuyuki and Watanabe, Shinzo (1981). Stochastic Dwerentid Equations and Dguvion processes. North Holland, Amsterdam.

Il’in, A. M., Kalashnikov, A. S., and Oleinik, 0. A. (1962). Linear quatione of the Kcond order of parabolic type. R u s . Math. Surwys 17.1-143.

1113, Kiyosi (1951). On stochastic differential equationr. Mem. A m . Math. Soc. 4. 116, Kiyosi and Watanabe, Shinto (1965). Transformations of Markov prowsacs by multiplicative

Jacod, Jean, MCmin, Jean, m d MCivier, Michd (1983). On tightness and stopping times. Stochas-

Acad. 40,262-266.

Appl. 1, 79-99.

line. Ann. Probab. 9,429-452.

Scand. J . Statist. 9, 79-94.

32-57.

4,443-477.

operator-valued Feynman-Kac formula. Comm Pure Appl. Math. 25,337-367.

Gebietr 30,3 15-320.

York.

Soc. Colloq. Publ. 31, Providence, R.I.

functionals. Ann. fmt. Fowler 15, 15-30.

tic Process. Appl. 14,109-146.

REFERENCES 513

Jagem, Peter (1971). Diffusion approximations of branching processes. Ann. Marh. Srarisr. 42,

Jifina, Miloslav (1969). On Feller's branching diffusion processes. Casopis Pbt. Mar. 94.84-90.

Joffe, Anatole and Metivier. Michel (1984). Weak convergence of sequences of semimartingales

Kabanov. Yu. M.. Lipster, R. Sh.. and Shiryaev, A. N. (1980). Some limit theorems for simple

Kac, Mark (1956). Some stochastic problems in physics and mathematics. Magnolia Petroleum

Kac, Mark (1974). A stochastic model related to the telegrapher's equation. Rocky M I . J. Math., 4.

Kallman, Robert R. and Rota, Gian-Carlo (1970). On the inequality l l f ' l l ' 5 411f11 11/"11.

Karlin, Samuel and Levikson, Benny (1974). Temporal fluctuations in selection intensities: Case of

Kato. Tosio (1966). Perturbation Theoryfor Linear Operarors. Springer-Verlag, New Yurk.

Keiding, Niels (1975). Extinction and exponential growth in random environments. Theor. Pop. Bid . 8,4963.

Kertz. Robert P. (1974). Perturbed semigroup limit theorems with applications to discontinuous random evolutions. Trans. Amer. Math. Soc. 199.29-53.

Kertz, Robert P. (1978). Limit theorems for semigroups with perturbed generators, with applications to multiscaled random evolutions. J . Funcr. Anal. 27.21 5-233.

Khas'minskii, R. 2. (1960). Ergodic properties or recurrent diffusion processes and stabilization of the solution of the Cauchy problem for parabolicequations. Theory Probab. Appl. 5, 179-196.

Khas'minskii, R. Z. (1966). A limit theorem for the solutions of difi'erential equations with random right-hand sides. Theory Probab. Appl. I I, 390-406.

Khas'minskii. R. Z. (1980). Srochasric Stability of Diflerenrial Equations. Sijthoff and Nordhofi',

Khintchine. A. (1933). Asymprorische Gesetze der Wahrscheinlichkeirschnuna. Springer-Verlag,

Kimura. Motoo and Crow, James F. (1964). The number of alleles that can be maintained in a

Kingman, J. F. C. (1967). Completely random measures. PaciJic J. Math. 21.59-78.

Kingman, J. F. C. (1975). Random discrete dislributions. J . R . Srarisr. SOC. B 37, I 22. Kingman, J. F. C. (1977). The population structure associated with the Ewens sampling formula.

Kingman, J. F. C. (1980). Marhematics of Genetic Diuersity. CBMS-NSF Regional ConJ Series in

Kolmogorov. A. N . (1956). On Skorohod convergence. Theory Pmhah. Appl. I , 215.222.

m74-207a.

with applications to multitype branching processes. Tech. Rep., Universitt de Montrtal.

point processes (a martingale approach). S~ochastics 3. M3-216.

co. Colloq. Lecl. 2.

497-509.

Inequalities. Vol. I / . Oved Shisha, Ed. Academic, New York. pp. 187- 192.

small population six. Theor. Pop. B i d . 6, 383412.

Berlin.

finite population. Generics 49, 725-738.

Theor. Pap. Biol. I I. 274-283.

Appf. Math., 34. SIAM, Philadelphia.

Komlos, JBnos. Major, Peter. and Tusnady, Gabor (1975. 1976). An approximation ol partial sums of independent RV's and the sample DF I, II . Z . Wahrseh. uerw. Gebiere 32, I 11-131 ; 34,33-58.

Krylov, N. V. (1973). The selection of a Markov process from a system of processes and the

Kunita, Hiroshi and Watanabe, Shinzo (1967). On square integrable martingales. Nagoya Math. J.

Kuratowski, K. and Ryll-Nardzewski, C. (196% A general theorem on selectors. Bull. Acad. Pofon.

construction of quasidiffusion processes. Marh. USSR Isuestia 7,691 -709.

30,209-245.

Sci. Ser. Sci. Marh. Astronom. Phys. 13, 397403.

514 RmRENcm

Kurtz, Thomas G. (1969). Extensions of Trotter's operator aemigroup approximation theorems. 1. Funct. Anal. 3,354-375.

Kurtz, Thomas G. (1970s). A general theorem on the convergoncc of operator scmigroups. Trans. Amer. Math. Soc. 148,23-32.

Kurtz, Thomas G. (1970b). Solutions of ordinary difTerentisl equations as limits of pure jump Markov proccsscs. J. Appl. Probob. 7,49-58.

Kurtz, Thomas G. (1971). Limit theorems for squenctr of jump Markov proasses approximating ordinary differential processes. J . Appl. Probab. 4 3 6 3 % .

Kurtz. Thomas G. (1973). A limit theorem for perturbed operator semigroup, with applications to random evolutions. J. Funct. AnaI. 12.55-67.

Kurtz, Thomas G. (1975). Semigroups of conditioned shifta and approximation of Markov processes. Ann. Prohob. 3,618-642.

Kurtz, Thomas G. (1977). Applications of an abstract perfurbation theorem to ordinary differen- tia! equations. Houston J. Math. 3,67-82.

Kurtz, Thomas G. (1978a). Strong approximation theorems for density dependent Markov chains. Stochastic Process. Appl. 6,223-240.

Kurtz, Thomas G. (1978b). Diffusion approximations for branching processes. Branching Pro- cesses, Adu. Proh. 5, Anatole Jofie and Peter Ney. Eds., Marcel Dekker. New York. pp.

Kurtz, Thomas 0. (198Oa). Representation of Markov proceues as multiparameter time changes.

Kurtz, Thomas G. (IY8Ob). The optional sampling theorem for martingales indexed by directed

Kurtz, Thomas G. (I98 la). Approximation of Population Processes. CBMS-NSF Regional Cod

Kurtz, Thomas G. (1981b). The central limit theorem for Markov chains. Ann. Probab. 9,557-560. Kurtz. Thomas G. (1981~). Approximation of discontinuous processes by continuous processes.

Stochastic Nonlinear Systems, L. Arnold and R. Lefever, Eds. Springer-Verlag. Berlin. pp.

Kurtz. Thomas G. (1982). Representation and approximation of counting processes. Advances in Filtering and Optimal Stochastic Control, Lett. Notes Cont. 1n.J Sci. 42, W. H. Fleming and L. G. Gorostiza. Eds., Springer-Verlag, krlin.

Kushner, Harold J. (3974). On the weak convergence of interpolated Markov chains to a diffusion. Ann. Probab. 2.40-50.

Kushner, Harold J. ( I 979). Pumpdiffusion approximation for ordinary differential equations with wide-band random right hand sides. S l A M J. Control Opfim. 17,729-744.

Kushner, Harold J. (1980). A martingale method for the convergence of a sequence of procespcv to a jump-diffusion process on the line. 2. Wahrwh. uerw. Gebiete S. 207-219.

Kushner, Harold 1. (1982). Asymptotic distributions of solutions of ordinary differential equations with wide band noise inputs; approximate invariant measures. Stockastics 6,259-277.

Kushncr, Harold J. (1984). Approximation and Weak Convergence Methods for Randm Procrsrcs. MIT Press, Cambridge, Massachusetts.

Ladyzhenskaya, 0. A. and Ural'tseva, N. N. (1968). Linear and Quasilinear Elliptic Partial D@r- eniial Equations. Academic, New York.

Lamperii, John (1967a). The limit of a sequence of branching proccsm. 2. Wohrsch. uerw. Gebiete

Lamperti. John (1967b). On random time substitutions and the Feller property. Markou Processes

Lamperti, John (1977). Stochastic Processes. Springer-Verlap, New York.

262-292.

Ann. Probab. 8,682-715.

sets. Ann. Probab. 8,675-681.

Series in Appl. Math. 36, SIAM. Philadelphia.

22-35.

7.271-288.

and forenrial Theory, Joshua Chover, Ed., Wiley, New York. pp. 87-101.

~ E N c E s 515

Uvy, Paul ( 1948). Processus Srochasrique er Mouuemenr Brownien. Gauthier-Villars, Paris.

Liggett, Thomas M. (1972). Existence theorems for infinite particle systems. Trans. Amer. Marh.

Liggett, Thomas M. (1977). The stochastic evolution of infinite systems of interacting particles.

Liggett, Thomas M. (1985). Interacting Parricle Sysrems. Springer-Verlag. New York.

Lindvall. Torgny (1972). Convergence of critical Galton-Watson branching processes. J . Appl. Probab. 9.445-450.

Lindvall, Torgny (1974). Limit theorems for some functionals of certain Galton-Watson branching proasses. Adu. Appl. Probab. 6,309-321.

Littler. Raymond A. (1972). Multidimensional stochastic models in genetics. Ph.D thesis, Monash Univ.

Littler, Raymond A. and Good, A. J. (1978). Ages. extinction times. and first passage probabilities for a multiallele diRusion model with irreversible mutation. Theor. Pop. Biol. IS. 214-225.

Mackevitius. V. (1974). On the question of the weak convergence of random processes in the space D[O, co). Lithuanian Marh. Trans. 14,620-623.

Maigret, Nelly (1978). ThCorhme de limite antrale functionnel pour une cheine de Markov recurnnte au scns de Harris et positive. Ann. Inst. Henri Poincark 14,425-440.

Major, Peter (1976). The approximation of partial sums of independent RV’s. Z. Wahrsch. uerw. Gebiete 35.2 I 3-220.

Malek-Mansour, M.. Van Den Brocck, C., Nicolis, G.. and Turner, J. W. (1981). Asymptotic properties of Markovian master equations. Ann. Phys. IJI.283-313.

Mann. Henry B. and Wald. Abraham (1943). On stochastic limit and order relations. Ann. Math. Sratisr. 14,217-226.

Mandl. Petr ( 1968). Analytical Treatment n/ One-Dimensional Marknu Processes. Springer-Verlag, Berlin.

McKean. Henry P., Jr. (IW9). Srochasric lnrrgrals. Academic, New York. McLeish. D. L. (1974). Dependent central limit theorems and invariance principles. Ann. Prohab.

MClivier. Michel (1982). Semimorringoles: A Course on Srochasric Processes. Walter de Gruyter.

Meyer. Paul-AndrC ( 1966). Prohahilily and Poienriols. Elaisdell, Waltham, Mass. Meyer, Paul-AndrC (1967). Inttgrales stochastiques I. II. 111. IV. Shinaire de Proba6iliti.s I . &cr.

Meyer, Paul-Andri (1968). Guide detaille de la theorie generale des processus. Skminaire do

Mirando. Carlo (1970). Porfial Diflerenrial Equations oj Elliptic Type. Springer-Verlag. Berlin.

Moran. P. A. P. (19588). A general theory of the distribution of gene frequencies 1. Overlapping

Moran. P. A. P. (1958b). A general theory of the distribution of gene frequencies. 11. Non-

Moran, P. A. P. (1958~). Random processes in genetics. Proc. Cam6. Phil. Soc. 51.60-71. Morando, Philippe (1969). Mesures aleatoim. Seminuire de Probobillres I I I . Lccr. Notes Marh. 88,

Morkvenas, R. (1974), Convergence of Markov chains to solution of martingale problem. Urh-

Nagylaki. Thomas (1980). The strong-migration limit in geographically structured populations. J.

Soc. 16!i,47 1-48 I.

k c r . Notes Marh. S!N. 187-248. Springer-Verlag, New York.

2.608-6 19.

Berlin.

Notes Marh. 39.72-162.

Probahiliris 11. Lccr. Nores Marh. 51. 140-165.

generations. Proc. Roy. Soc. London B 149, 102-1 11.

overlapping generations. Proc. Roy. Sw. London B 149. I 13-1 16.

190-229. Springer-Veda& Berlin.

uanian Math. Trans. 14,460-466.

Marh. Bid . 9, 101-1 14.

516 REFERENCES

Nagylaki, Thomas (1982). Geographical invariance in population genetics. J . Theor. Elol. 99,

Neveu, J. (1958). Thtorie des semi-groups de Markov. Uniu. Culi-fwniu Publ. Srarlst. 2,319-394.

Norman, M. Frank (1971). Slow learning with small drift in two-absorbing-barrier models. J .

Norman, M. Frank (1972). Markou Processes and Learning Modrls. Academic, New York. Norman. M. Frank (1974). A central limit theorem for Markov processes that move by small

Norman, M. Frank (1975s). Diffusion approximation of non-Markovian proasses. Ann. Probab.

Norman, M. Frank (1975b). Limit theorems for stationary distributions. Adv. Appl. Probab. 7,

Norman, M. Frank (1977). Ergodicity of diffusion and temporal uniformity of diffusion approximation. J. Appl. Probab. 14,399-404.

Ohta. Tomoko and Kimura, Motoo (1969). Linkage disequilibrium due to random genetic drik Genei. Res. Camb. 1347-55.

Ohta, Tomoko and Kimura. Motoo (1973). A model of mutation appropriate to estimate the number of electrophoretically detectable alleles in a finite population. Gem. Res. Comb. 22. 201-204.

Oleinik, Olga A. (1966). Alcuni risultati sulle equazioni lineari e quasi lineari ellitico-paraboliche a derivate parziali del second ordine. Atd Accad. Not. Uncei Rend. Cl. Sci. Fis. Mac. N o w . (8) 40,775-784.

Papanicolaou, George C. and Kohler, W. (1974). Asymptotic theory of mixing stochastic ordinary differential equations. Comm. Pure Appl. Math. 27.641668.

Papanicolaou, George C., Stroock. Daniel W., and Varadhan, S. R. S. (1977). Martingale approach to some limit theorems. Conference on Statisrical Mechanics, Dynaniicai Sysrems. and Turbulence, Duke Uniwrsity. M. Red, Ed., Duke Univ. Math. Series 3.

Papanicolaou, George C. and Varadhan, S. R. S. (1973). A limit theorem with strong mixing in Banach space and two applications to stochastic diRenntial equations. Comm. Pure Appl. Math. 26,491-524.

159-1 72.

Math. Psych. 8, 1-21.

steps. Ann. Probab. 2.1065-1074.

3.358-364.

561-575.

Parthasarathy, K. R. (1967). Probability Measures on Metric Spaces. Academic, New York. Pazy, Amnon (1983). Semigroups oj Linear Operarors and Applications 10 Partial Differential Equa-

Peligrad, Magda (1982). Invariance principles for mixing sequences of random variables. Ann.

Phillips, Ralph S. and Sarason, Leonard (1968). Elliptic-parabolic equations of the second order. J . Math. Mech. It, 891-917.

Pinsky, Mark A. (1968). DiDrential equations with a rmall parameter and the central limit theorem tor functions defined on a finite Markov chain. Z. Wahrsch. uerw. Gebiete 9, 101-1 11.

Pinsky, Mark A. (1974). Multiplicative operator functionals and their asymptotic properties. Aduances in Probability 3, Dekker, New York.

Priouret, P. ( I 974). Processus de diffusion CI equations difftrentielles stochastiques. Lea. Nores Marh. 390. Springer-Verlag. Berlin.

Prohorov, Yu. V. (1956). Convergence of random processes and limit theorems in probability theory. Theory Probab. Appl. I, 157-214.

Rao. C. Radhakrishna (1973). Linear Starisrical Iqference and l r s Applications, 2nd ed. Wiley, New York.

Rebolledo, Roland0 (1979). La mbthod des martingales appliquk P I'ttude de la convergence en loi de processus. Euli. Soc. Marh. France Mem., 62.

r ims . Springer-Verlag, New York.

Probab. 10,968-981.

REFERENCES 517

Rebolldo, Rolando (1980). Central limit theorems for local martingales. 2. Wahrsch. wrw.

Rishel. Raymond (1970). Necessary and suficient dynamic programming conditions for contin-

RootECn, Holger (1977). On the functional central limit theorem for marlingaler. 2. Wahrsch. uerw.

Rootdn. Holger (19RO). On the functional antral limit theorem for martingales I I . 2. Wahrsch.

Rosenblatt, Murray (1956). A central limit theorem and a strong mixing condition. Proc. Nut.

Rosenkrant% Walter A. (1974). Convergent family of diffusion proasses whose difrusion coetli-

Rosenkrantz, Walter A. (1975). Limit theorems for solutions to a class of stochastic differential

Rosenkrantz, Walter A. and Dora, C. C. Y. (1980). Limit theorems for Markov processes via a

Roth. Jean-Pierre (1976). O p t a t e u n dissipatifs et semigroups dans les espacts de fonctions con-

Roth. Jean-Pierre (1977). Opirateurs elliptiques comme gtntrateurs infinitbsimaua de semi-

Rudin, Walter (1973). Functional Analysis. McGraw-Hill. New York. Rudin, Walter (1974). Real and Complex Analysis. 2nd Ed. McGraw-Hill, New York. Sato, Ken-iti (1976). Diffusion processes and a class of Merkov chains related to population

Sato. Ken-iti (1977). A note on convergence of probability measures on C and D. Ann. Sd .

Schauder. J. (1934). Uber lineare elliptische Difrerentialgleichungen zweiter Ordnung. Marh. 2. 38,

Schliigl, F. (1972). Chemical reaction models for non-quilibrium phase transitions. Z. Physik 253,

Scrant, Daniel and Villard, Miehel (1972). Linearization of crossing-over and mutation in a finite

Shiga. Tokuzo (1980). An interacting system in population genetics. J. Morh. Kyoro Uniu. 20.

Shiga. Tokuzo (1981). Difrusion processes in population genetics. J . Marh. Kyoro Uniu. 21. 133-

Shiga, Tokuzo (1982). Wandering phenomena in infinite allelic diliusion models. Adu. Appl.

Siegmund. David (1976). The equivalence of absorbing and reflecting barrier problems for stochas-

Skorohod, A. V. (1956). Limil theorems for stochastic processes. Theory Probab. Appl. 1,261 -290.

Skorohod, A. V. (1958). Limit theorems for Markov processes. Theory Probob. Appl. 3.202-246. Skorohod, A. V. (1965). Srudfes in rhe Theory nj’ Random Processes. Addison-Wesley, Reading.

Sova. Miroslav (1967). Convergence d’optrations IinCaires non borntes. Rev. Rnumaine Marh.

Spitzer. Frank (1970). Interaction of Markov proasses. Adu. Marh. 5, 246--290.

Gebiere SI, 269-286.

uous time stochastic control. S I A M J. Conrrol 8.559-571.

Gehiere 3, 199-210.

uerw. Gebiere 5I,79-93.

Acad. Sci. U S A 42.4347.

cients diverge. Bull. Amer. Math. Soc. 80.973 976.

equations. Indiana Uniu. Marh. J . 24.613-625.

variant of the Trotter-Kato theorem. J. Appl . Prohab. I t . 704-715.

tinues. Ann. I n n . Fourier, Grenoblr 26, 1-97.

groupes de Feller. C. R. Acad. Sci. Paris 284, 755-757.

genetics. Osaka J . Math. 13,631-659.

Kanazawa Uniu. 14. 1-5.

257-282.

147-161.

random-mating population. Theor. Pop. Bid. 3,249-257.

21 3-242.

151.

Prohab. 14.457483.

tically monotone Markov processes. Ann. Probab. 4.9 14-924.

Mass.

Pures Appliq. 12. 373-389.

518

Stone, Charles (1963). Weak converpence of slochmtic proaucr defind on a semi-infinite time interval. Proc. Amer. Math. Soc. 14,694-696.

Stratonovich. R. L. (1963, 1967). Topics in the Theory qflandom Nohe 1. /I. Gordon and Breach, New York.

Sirassen, Volker (1965). The existence of probability measurea with given marginals. Ann. Math. Statist. 36423-439.

Stroock, Daniel W. (1975). Diffusion processes auociatd with Levy generators. 2. Wahrsch. wrw. Gebietc 32,209-244.

Stroock. Daniel W. and Varadhan, S. R. S. (1969). DitTusion processes with continuous d i c i e n t s I, 11. Comm. Pure Appl. Math. 22,345-400.479-530.

Stroock, Daniel W. and Varadhan, S. R. S. (1971). Dilfurion p m r s n with boundary conditions. Corn. Pure Appl. Math. 24, 147-225.

Stroock, Daniel W. and Varadhan. S. R. S. (1972). On the iupport of ditTusion proaases with applications to the strong maximum principle. Proc. Sixth lkrkcley Symp. Math. Statist. Prob. 3,333-359.

Stroock. Daniel W. and Varadhan. S. R. S. (1979). Multidimensional Di@usion Processes. Springcr- Verlag. Berlin.

Trotter, Hale F. (1958). Approximation of semi-group of operaton. P a @ 1. Math. 8,887-919. Trotter. Hale F. (1959). On the product of Semi-group of operators. Proc. Amer. Math. Soc. 10,

Vasershtein, L. N. (1969). Markov processes over denumerable producu of spaces describing large

Vasershtein, L. N. and Leontovich, A. M. (1970). Invariant measures of certain Markov operators

Volkonskii, V. A. (1958). Random substitution of time in strong Markov proasses. Theory

Volkonskii, V. A. and Rozanov, Yu. A. (1959). Some limit theorems for random functions 1.

Wagner, Daniel H. (1977). Survey of measurable selection theorems. SIAM J. Contrd O p t k . IS,

Wan& Frank J. S. (1977). A central limit theorem for age and density dependent population

Wan& Frank J. S. (1982a). Probabilitica of extinction of multiplicative measure diffusion processes

Wang, Frank J. S. (l982b). Diffusion approximations of age-and-position dependent branching

Watanabe, Shinzo (1964). Additive functionah of Markov procer#s and U v y systems. Japanese J.

Watanabe, Shinzo (1968). A limit theomn of branching processes and continuous state branching

Watanabc, Shinzo (1971). On stochastic differential equations for multi-dimensional diffusion

Wattenon, 0. A. (1962). Some thcoreticol aspecta of ditTusion theory in population wnetics. Ann.

Watterson, G. A. (1964). The application of diffusion theory to two population genetic models of

Wattcison. G. A. (1970). On the equivalence of random mating and random union of gametes in

545-551.

systems of automata. Probl. Pcredachi IMorm. 3(3),64-73.

describing a homogeneous random medium. P r d . P d a c h l /dorm. ql). 71-80.

Probab. Appl. 3.3 10-326.

Theory Probab. Appl. 4.178-197.

859-903.

processes. Stochastic Pronss. Appl. 5, 173-193.

with absorbing boundary. Indiana Unlu. Math. 1.31.97-107.

processes. Stochastic Process. Appl. 1359-74.

Math. 34,53-70.

processes. J. Math. Kyom Uaiu. 8,141467.

processes with boundary conditions. 1 . Math. Kyoto Uniu. 11,169-380.

Math. Statist. 33,939-957.

Moran. J. Appl. Probab. 1.233-246.

finite, monoecious populations. Theor. Pop. Bid. 1.233-250.

REFERENCES 519

Watterson, G. A. (1976). The stationary distribution or the infinitely-many neutral alleles diflusion

Watteraon. G. A. and Guess, Harry A. (1977). Is the most frequent allele the oldest? Theor. Pop.

Weiss. Alan (1981). Invariant measures of diffusions in bounded domains. Ph.D. dissertation. New

Whitney, Hasskr (1934)). Analytic extensions of differentiable functions defined on closed sets.

Williams. David (1979). Difusions, Markou Processes, and Morringales. Wiley. New York. Withers, C. S. (1981). Central limit theorems for dependent variables. 1. 2. Wahrsch. uerw. Gebicre

Wonham, W. M. (1966). Lyapunov criteria for weak stochastic stability. J. Diferenrial Equarions t

Wright. Sewall (1931). Evolution in Mendelian populations. Generics 16,97-159. Wright, Sewall (1949). Adaptation and selection. Gtnetics. Paleonfology. and Euolurlon, G . L.

Jepson, E. Mayr, and G. G. Simpson. Eds. Princeton University Press, Prinaton. pp. 365- 389.

Yamada, Toshio and Watanabe. Shinzo (1971). On the uniqueness of solutions of stochastic diflerential equations. J . Math. Kyoro Uniu. 11. 155-167.

Yosida, Kosaku (1948). On the differentiability and the representation of one-parameter semipoups oflinear operators. J. Math. Soc. Japan 1. IS21

Yosida. Kosaku (1980). Puncrional Analysis. 6th ed. Springer-Verlag. Berlin. Zakai, Moshe (1969). A Lyapunov criterion for the existence of stationary probability distribu-

tions for systems perturbed by noise. S l A M J . Conrro/7,390-397.

model. 1. Appl. Prohb. 13,639451.

Biol. 11. 141-160.

York University.

Trans. Amer. Math. SOC. 36,369-387.

57,509-534.

195-207.

INDEX

Note: * indicates definition

A. 23+ .'/ , 145* Abraham, 499 Absorption probabilities, convergence, 420 Absorption lime. convergence in distribution,

419 Adapted:

to dimted filtration, 86. stochasric process, 50*

Aldous, 154 Alexandroff. IS4 Allah. 467 A h , 467 Analytic sets. 506 Anderson. 273 Artstein, 154 Athnya, 409

@E), ISS* .+?, 96* Barbour. 409.467 BBrtfAi. 359. 364. 464 BeneJ, 274 Bhattacharya. 364 Billingsley. 154, 364. 445. 504 Bimeasure. 502 Blackwell. 154 Blankenship. 274 Bochner integral, 473* Borovkov. 364 Boundary classification. one-dimensional

diffusion, 366, 382

Boundary of set. IOR* Bounded generator, 162. 222 Bounded pointwise convergence. I I I+. 495 bp-closure, I I I *, 496* bp-convergence, 1 1 t * bpdense. I I I * bplim, 495* Branching Markov prncess. 400

genetaror. 402 Branching process:

Galton-Watson. 386' in random environments. 396 two-type Markov. 392

Bmwn, 364 Bruwnian motion, 276'. 302. 359

martingale characterization. 290, 338 stmng approximation. 356

Cmu(f l l , 368 Castaine. 506 C(E). 164* CfE), 155, CJ0, ") niartingale problem, IR6* Chapman-Kolmogomv pmpetiy. 156' Chemical reaction model. 2. 454. 466 Chentov. 154 Chernoff. 48, I54 Chemoff inequality, 30 Chemoff product formula. 32 Chow, 94 Chung. 305 Closed linear operator. 8* ClosuR of linear operator. 16'

52 1



522 INDEX

Compact containment condition. 129, for Schlogl model. 4 suftlcient conditions, 203

Compact XIS:

in DeIO. OD), 123 in.rO(S). 104

Conservative operator, 166' Contact time, 54' Continuous mapping theorem. I03

Convergence determining set. I I2* generalization, I5 I

conditions for, 151 counterexample, I51 on product space, I I5

for branching Markov processes, 406 via convergence of generators, 2. 388, 393. 406. 415. 428, 436,475, 480. 484

in DEIO, a). 127. 131. 143. 144 for Feller processes. 167 for Markov chains. 2. 3, 168. 173. 230. 233.

for Markov processes, 172. 230. 232. 236 for measure-valued genetic model, 439 to process in CJO, a), 148 wing random time change equation, 3. 310. 322, 323, 390, 397.458

90

Convergence in distribution, 108*

236

Convergence in probability, metric for, 601,

Core, 17* conditions for. 17. 19 examples, 3, 43. 365 of generator, 17* for Laplacian, 276

Costantini. 5. 274 Courr&ge. 305 Cox. 274 Crandall. 47 Crow, 451 csikgo, 364

DEIO, m), 116' Bowl sets in. 127 compact sets in, 122 complelemss. 121 modulus of continuity, 122*. 134 separability. 121

Darden, 467 Davies, 47, 48 Davydov. 364 Duwson. 274. 409 Delfacherie, 73, 73, 93. 94, 507 DJO. a) martingale problem. 186'

uniqueness, 187

Density dependent family, 455. diffusion appmximation, 460 Gaussian approximation. 458 law of large numbers. 456

Diffusion approximation: density dependent family, 460 examples, I, 360, 361 Galton-Watson process, 388 genotypic-frequency model, 426 for Markov chain. 355, 428 maningale pmblem, 354 random evolution. 475, 480,484 random time change equation, 330 stmng approximation. 460 for Wright-Fisher model. 363.415

absorbing boundary. 368 boundary classification, 366, 382 degenerate, 371 generator, 366 one-dimensional, 367 in R', 370 random time change equation, 328 reflecting boundary. 369 stochastic integral equation for. 290

Discontinuities of functions in 410, a): convergence of, 147 countability of, I16

Discrete stopping time: approximation by, 86 strong Markov propeny. 159

of linear operator, I I * mutingale problem. 178 of multivalued linear operator. 21 positive maximum principle, 165*

convergence in. 108' of random variable. 107.

Doleans-Dade. 94.305 Dominated convergence t k r c m . 492 Donsker. 364 Doob. 93. 94.273, 305. 364. 478 Doob inequality, 63. 64 Doob-Meyer decomposition, 742 Driving process. 469 Duality, 188*. 266 Dubins. 154 Dudley, 154 Dunford. 494 Dvoretzky. 364 Dynkin. 47. 93. 273. 370. 385 Dynkin class. 496, Dynkin class theonm. 497

Diffusion process:

Dissipativity:

Dislribution:

INDEX 523

ES, 165, Echevenia. 274 Elliot. 75, 305 Entrance time, 54+ Epidemic model. 453, 466 Equivalence of stochastic pmcesses. 50* Ergodic theorem. 352 Ethier. 44. 48. 274. 371. 372. 375. 385, 451 Ewens, 45 I Ewens sampling formula, 447 Exit time. 54,

ex-LIM, 34. ex-lim. 22' Extended limit. 22'

generalized, 34

convergence in distribution,n. 419. 464

q,( 'I 1 )/ ), 346 .??, 50+ Fatou's lemma, 492 Feller. 116. 385. 409 Feller process;

continuity of sample paths. 171 convergence in distribution, 167 version in D,lO, 4, 169

for Brnwnian motion, 276

complete, 50* directed index set. 85 Markov with respect to. 156 right continuous, 50,

Finite-dimensional distributions convergence, 225. 226 detennincd by semigmup. 161 for Markov process. 157 of stochastic process. 50*

Feller semigroup. 166*

Filtration, SO+

Fiorcnta. 385 Fisher, 451 Fleming, 45 I Fleming-Vior model, 440. 450

martingale problem, 436 Forward equation, uniqueness, 25 I. 252 Freidlin. 385 Friedman. 305. 369 Full generator, 23*. 261

related martingales, 162

Galton-Watson branching process. 386 Gfnrler. 364 Generator:

absorbing diffusion, 36R bounded, 162. 222 bounded perturbation, 256. 257, 261

branching Markov pmcess. 402 core of. 17+ d-dimensional diffusion, 370. 372, 373. 374,

degenerate diffusion. 371, 372, 373, 374, 375.

examples, 3, 43, 365 extended limit. 22'. 34 full, 23. Hille-Yosida theorem. 13 independent increments process. 380 infinite particle system, 381 jump Markov pmcess. 376 Lcvy process, 379 nondegenerate diffusion, 366. 370 one-dimensional diffusion. 366, 371 perturbation. 37. 44 properties, 9 reflecting diffusion. 369 resolvent of, 10+ of semigroup. 8' uniqueness of semigmup, I 5 Yosida approximation of, I2+

Genotypic-frequency model, 426 Genrdi, 274 Gihman. 305, 380 Goldstein, 48, 491 Good. 451 Gny. 274 Griego. 491 Griffeath, 274 Grigelionis. 364 Grimvall, 409 Oronwall's inequality. 498 Guess, 451 Gustafson. 48

375

408

Hall, 154, 364 Hardin, 364 Hardy- Weinberg pmpcwtions, 4 12'

Hams, 273 Hasagawa. 48 Hlusler. 364 Helland, 274. 335, 364. 409 Helms, 336 Hersh, 491 Hey&, 364 Hille. 47. 48 Hille-Yosida theorem. 13, 16

deviation from. 43 t

for multivalued generators, 2 I for positive semigroups on e, 165

Himmelberg. SO6 Hitting distribution. convergence. 420. 464

524 INDEX

Hochberg. 409 Holley. 273, 274. 336

Ibragimov. 364 Ikeda, 305,409 Win, 385 Increasing pmcess, 74+ Independent increments pmess, genentor, 380 Indistinguishability of stochrstic processes. 50* Infinitely-many-allele model, 435

convergence in distribution. 436 Infinite particle system. generator, 381 Infinitesimal generator (sce generator) Initial distribution. 157* Integral, Banach space-valued. 8, 473 Integration by parts, 65 Invariance principle, 278

for stationary sequence. 350 strona. 356

Inverse function. measurability of, 506

Itb's formula, 287 ita,93,305

Jacod. 154 lagers. 409 Jensen's inequality, 55 JiHna, 409 Joffe. 409 lump Markov process:

construction. 162, 263 examples, 452 generator, 376 random time change equation. 326,455

Kabanov, 364 Kac, 491 Kalashnikov, 385 Kallman, 48 Karlin. 450 Kato, 48 Keiding. 409 Kenz, 48 Khas'minskii, 274, 491 Khintchine. 274 Kimura, 451 Kingman, 451, 502 Kohler, 491 Kolmogorov, 154 Komlds, 356.364.459 Krylov, 274 Krylov's theorem, 210 Kunita, 305 Kuratowski, 506

Kurt~, 5.47. 48,94, 154. 273. 274, 336, 364.

Kushner, 4, 274, 364, 491

Ladyzhenskryr. 369, 385 Lamperti, 335, 352,409 Leontovirch, 273 Levitson, 450 Uvy, 273, 364 Levy process. generator, 379 Ligeett, 47. 48. 273. 381.385 LindvaII. 154, 409 Linear operator, 8.

closable, 16* closed, 8+ closure of,' 16* dissipative, I I * graph of, 8* multivalued, 20 single-valued, 20

409,451, 467, 491

Linnik, 364 Lipster, 364 Litrler, 385. 451 L'Wh 280 Local martingale, 64'

example, 90 see 01so Martingale

Local-martingale problem, 223+ Logistic growth model. 453, 466 Lyapunov function, 240

McKean, 305 Mackevitius. 273 McLcish, 364 Maigret, 364 Major, 356, 364, 459 Malek-Mansour. 5 Mandl, 367. 385 Mann, 154 Markov chain, 158*

Markov process. 156' diffusion approximation. 355, 428

convergence in distribution. 172, 230, 232.

corresponding semigroup. 161 sample paths in CJO, *), 264. 265 sample paths in DJO. 0). 264

236

Marriage lemma, 97 Maningale, 55*

central limit theorem. 339. 471 characterization using stopping times, 93 class DL. 74, continuous. 79 Convergence in disrriburion. 362

INDEX 525

MikuleviCius. 364 Mixing, 345. 362 Modification:

progressive. 89 of stochastic process. 50*

93 Modulus of continuity, 277* in DdO. 9. 122*, 134. 310, 321, 334

Monotone class. 496' Monotone class theorem, 496

for functions. 497 Moran, 45 I Moran model. 433 Morando, 502 Morkvenas, 274 Multiparameter random time change, see

Random time change equation Multiparameter sropping rime, 3 I 2 Multivalued operator, 20*

domain and range of, 20*

Nagasawa, 409 Nagylaki. 48. 449. 451 Nappo, 5 . 274 Neveu. 47 Ney. 409 Norman, 48. 274, 385. 451, 467

Offspring distribution, 386 Ohta. 451 Ohta-Kimura model, 440. 450 Oleinik, 374. 385 Optional:

modification, 72 process, 7 I sets, 71*

Optional projection, 73+ in Banach space, 91

optional projection theorem. 72 optional sampling theorem. 61, 92, 93

directed index set. 88 multiparameter. 317

Omstein-Uhlenbeck process, 191

Papanicolaou. 274, 491 Patthasamthy. 506. 507 Pazy, 47 P-continuity set. 108. .+YE). 96 Peligrad, 364 Perturbation by bounded operator, 38 Perturbation of generator, 37. 44 Phillips, 47. 385 Picard iteration. 299 Pinsky. 491

cross variation. 79 directed index set, 87. Doob inequality, 63, 64 local. 64 multiparameter, 3 I7 oplional sampling theorem, 61, 88. 92, orthogonal, 80. ( ) process. 79*, 280. 282 quadratic variation. 67*. 71 relative compactness, 343 right continuity of. 61 sample paths of, 59, 61 square integrable. 78'. 279 upcmssing inequality for, 57

bounded perturbation, 256. 257 branching Markov process. 404 CJO, m), 186* collection of solutions. 202 continuation of solutions. 206 convergence in distribution, 234

diffusion approximation. 354 discwe rime, 263 for distributions, 174. equivalent formulations, 176 existence. 199, 219, 220 existence of Markovian solution, 210 independent components. 253 local. 223, localization. 2 16 Markov property. 184 measure-valued ~ N X C S S , 436 for processes. 173' for random time change, 308 sample path properties. 178 sample paths in CJO. =), 295 stopped. 216* Schl6gl model. 3 time-dependent . 22 I uniqueness, 182.. 184. 187. 217, 219

Martingale problem, 173'

DJO. XI), 186*

well-posed. 182. Matrix square root. 374 W E ) . 155, Measurability of P,, 158. 188, 210 Measurable selection, 505 Measurable semigroup. 23' Measurable stochastic process, SO*

Mtmin, 154 Mttivier. 154, 30S, 409 Metric lattice, 85'

Meyer, 73, 75, 93, 94. 303, 507

MeaWR-valued process, 401, 436

separable from above, 8S

526 INDEX

Poisson pmcess. maningale characterization, 3 0 Ponm;mtcau themm. 10s Positive maximum principle. 165* Positive operafor. 165+ Positive semigroup. 165* Prdicublc pmcess, 75 Rioumt. 305 Process, see Stochastic pmcess Roduct space:

separating and convergence determining sets

tightness in. 107 Rogmssive modifcation. 89 h g m s i v e sets. 71.

with directed index ret, 86 Progressive stochastic process, 50' Rohomv, 154 Pmhmv metric, 96. 357.408

in. 115

completeness of, 101 separability of, 101

Prohmv theorem, 104

Quadratic vm'ation of local mmingafe, 67 Quasi-left continuity, 181

Random evolution, 469* diffusion approximation, 475, 480. 484

Random time change, multipammeter. 31 I Random time change equation, 306.

convergence in distribution. 310. 322. 323.

corresponding mMingak problem, 308, 309,

comsponding stochastic integral equation. 329 diffusion approximation, 330 for diffusion process, 328 for jump Markov process. 326,455 rnultipsnmeter, 312, nonanticipating solution. 314. 315 nonuniqueness. 332 relative compactness, 32 I for Schltlgl model, 3 stmng uniqueness. 314 uniqueness, 307 weak solution, 313 weak uniqueness, 314

390. 397.458

316

Rao. 478 Rebolledo, 274. 364 Relative compaciness:

in 410. 0). 197. 343 in .9lDJO, a)), 128, 137. 139, 142. I52

Resolvent identity, I I Resolvent for semigmup. 10. Resolvent set. lo*

Reversibility. 450+ Rtvtaz. 364 Rishel, 94 Robbin, 499 RooczCn, 364 Roscnblatt, 364 Rmnkrantz, 274 Rbler. 274 Rota, 48 Roth, 274. 373. 385 Rozanov. 364 Rudin. 4% Ryll-Ndzewski, 506

Sample paths: continuity of, 171 for Feller process. 167 for solution of martingale problem, 178 of stochastic process. SO*

Samson, 385. Sato, 154, 451 Schaudcr, 385

Schlbgl model, 2 Schwartz, 494 Scmigmup. 6,

45,46

Schldgl, 5

approximation theorem, 28, 31, 34, 36, 39,

with bounded generator, 7 of conditioned shifts, 80+. 92. 226, 229. 485 contraction, 6. convergence, 225. 388 convergence of resolvents, 44 corresponding to a Markov process. 161 ergodic pmpetties, 39 Feller, IW+ generator of, 81 Hille-Yarida theorem. 13 for jump Markov pmccss, I63 Jimif of pcnurbed, 41. 45.473 8nCSuNble. 23.. 80

positive. 165' strongly continuous. 6+ unique determination, I5

Separable from above, 85' Separating set (of functions). I 122

on product space, I I5 on subset of.flS), I16

Separation (of points), 112' Scrant. 385 Set-valued functions, measurability. 506 Ship. 274, 451 Shiryuv, 364

penurbetion, 37

INDEX 527

Siegmund, 273 Single-valued operator. 20* Skorohod. 47, 154. 274. 305. 364. 380 S k d o d representation. 102

Skomhod topology: inR. 150

compact sets in, 122 compkteness, I2 I metric for, 117.. 120. separability. I21

Slutsky. IS4 Slutsky theorem, I10 Sova. 47. 48 Space-time pmcess. 221, 295 Spicter. 273 Stationary distribution, 238'. 239

characterization, 248 convergence. 244. 245. 418 existence, 240. 243 for genetic diffusion. 417, 448 infinitely-many-allele model. 443 relative compactness, 246 uniqueness. 270

Stationary pmccss. 238* Stationary sequence:

i n v d m e principle for. 3SO Poisson approximation, 362

Stieltjes integral. 280 Stochastic integral:

iterated. 286. 287 with respect to local martingale. 286* with respect to martingale, 282' for simple functions, 280

msponding martingale problem, 292. 293 comsponding random time change equation.

existence, 2Qp, 300 pathwise uniqueness. 291. 2%. 297. 298 uniqueness in distribution, 291, 295. 296

adapted, SO* (right. left) continuous. SO* quivalence of, 50* finite-dimensional distributions. SO* increasing. 74' index set of. 49' indistinguishability, 50* measurable. SO* modification of, SO* progressive. 50' sample paths of, 50* state space of, 49. version of, 50*

Stochastic integral equation, 290

329

Stochastic process. 49*

Stone. IS4 Stopped martingale problem, 216' Stopped pmcess, X', 64.68, 285 Stopping time. SI*

approximation by discrete, S I, 86 bounded. 51. closure properties of collection. S I contact time. 54. corresponding a-algebra. 52*, 89 directed index set, 85 discrete. 51, enmce time, S4* exit time. 54. finite, 51' buncation of, 5 I , 86

Smssen, IS4 Stratonovich. 491 Strong approximation. 356, 460 ~ t m g Markov process, isa* Strong Markov property, IS8*

for Brownian motion. 278 Strong mixing. 345. Stmng separation (of points), I13*, 143 Stmock, 273, 274, 305. 336. 364, 369. 371.

Submartingale, SS* of class DL. 74.

Supermartingale. SS* nonnegative, 62

374, 375. 380, 385

Telegrapher's equation, 470 Tightness. 103. Time homogeneity. 156. Total variation norm, 495. Transition function:

continuous time, 156. discrete time, 158. existence. 502

Trotter. 47, 48. 274, 451 Trotter pruduct formula. 33

alternative pmof. 45 Tulaa's theorem. 504 TusJdy. 356. 364.459

Uniform integrability. 493. of class DL submartingale, 74 of conditional eapectations. 90 of submartingales, 60, 90 weak compactness. 76

Uniform mixing, 345*. 348, 484 Uniqueness:

for forward equation. 25 I, 232 for mcutingale problem, 182 for random time change equation. 307. 314

528 INDEX

Uniqueness (Coniinued) for stochastic integrml equation, 291. 295. 2%.

foru' * Au, 18. 26 Upcrossing inequality, 57 Ural'tscva, 369, 385

297, 298

Varadhan, 273. 274. 305. 364, 369, 371. 374,

Vasershtein, 273 Version of stochastic process, 50+ Villard. 385 Viot, 451 Volkonski. 335. 364

375, 385.491

Wagner, 506 Wald, 154 Wang, 409, 467 Watanak. 47, 93, 273. 305. 364. 385,409 Wattemon, 450. 45 I

Weak convergence, 107*. See also Convergence

Weak topology, metric for, %, 150* Weiss, 274 Whitney, 499 Whitney extension theorem, 499 Williams, 273, 274, 305 Withers, 364 Wonham, 274 Wright, 451 Wright-Fisher model, 414

in distribution

X'. 64*

Yamada, 305, 385 Yosida. 47 Yosida approximation of generator, 12*. 261

Zakai. 214

FLOWCHART

This table indicates the relationships between theorems, corollaries, and so on. For example, the entry C2.8 P2.1 P2.7 T6.9 T4.2.2 under Chapter 1 means that Corollary 2.8 of Chapter 1 requires Propositions 2.1 and 2.7 of that chapter for its proof and is used in the proofs of Theorem 6.9 of Chapler 1 and Theorem 2.2 of Chapter 4.

Chapter 1

P1.l C1.2 P1.56 C1.6 C1.2 P1.l R1.3 L1.40 Prob.3 L1.4b Prob.3 P2.1 P3.4 L 1 . 4 ~ Prob.3 P1.5~ 0 . 5 P5.4 L6.2 P1.Sa C1.6 P1.5b Pl.1 P2.1 P3.3 P3.7

T10.4.1 PlSc L1.4~ C1.6 T2.6 P2.7 P3.4 L6.2 T6.11 R4.2.10 P4.9.2 T8.3.1 C1.6 P1.l P1.5ac P2.1 T2.6 P2.7 P4.9.2 T10.4.1 P2.1 L1.4b L1.5b C1.6 T2.6 P2.7 C2.8 P3.3 P3.7 P4.1 T6.1 L6.3 T6.5 T6.9 77.1 RZ9b T2.7.1 C4.2.8 L2.2 L3.3 T2.6 L2.11 L2.3 L2.2 T2.6 T4.3 T6.9 T6.11 T4.5.19a L2.40 T2.6 P2.7 T6.1 L2.4b T2.6 P2.7 L2 .4~ T2.6 P2.7 T6.1 C6.8 T7.1 L2.5 L1.4c T2.6 P2.7 T2.6 P1.5~ C1.6 P2.1

L2.2 L2.3 L2.4abc L2.5 T2.12 P3.4 T4.3 V . 1 T4.4.1 P2.7 P1.5~ C1.6 P2.1 L2.4abc L2.5 P2.9 C2.8 T6.1 T4.2.7 C2.8 P2.1 P2.7 T6.9 T4.2.2 P2.9 P2.10 P2.7 P2.10 P2.9 P3.4 L2.11 L2.2 T2.12 P3.1 P3.4 12.12 T2.6 L2.11 P3.1 P3.5 P3.7 T4.2.2 P3.1 L2.11 T2.12 P3.3 T6.1 L6.3 T6.5 R3.2 T8.1.5 T8.3.1 P3.3 P1.5b P2.1 P3.1 P5.1.1 T8.1.6 T8.2.1 T8.3.1 T8.3.4 L10.3.1 P12.2.2 P3.4 L1.4b P1.5~ T2.6 P2.10 L2.11 P3.5 T2.12 L3.6 P3.7 T8.3.1 P3.7 P1.5b P2.1 T2.12 L3.6 C3.8 T8.2.1 TB.2.5 T8.2.8 C3.8 P3.7 P4.1 P2.1 T7.1 RZ9c T12.2.4 L4.2 T4.3 T4.4.1 T12.4.1 14.3 L2.3 T2.6 14.2 T8.3.1 P5.1 C4.8.7 C4.8.16 P5.2 P2.7.5 P5.3 P5.4

L1.4c T9.4.3 R5.5 T6.1 P2.1 L2.4ac P2.7 P3.1 L6.2 L6.3 T6.5 T7.6a C7.7a T4.2.5 T4.2.11 R4.8.8a T8.3.1 L6.2 P1.4~ P1.5c T6.1 L6.3 P2.1 P3.1 T6.1 L6.4 T6.5 T6.11 T6.5 P2.1 P3.1 L6.4 T6.1 C6.6 C6.7 C6.8 77.66 C7.7b T4.2.6 T4.2.12 T5.1.2~ T9.1.3 T10.1.1 C6.6 16.5 C6.7 C6.8 C6.7 T6.5 (26.6 C6.8 L2.4~ T6.5 C6.6 T2.7.1 T4.4.1 P4.9.2 T4.9.3 T6.9 P2.1 L2.3 C2.8 T6.11 T4.8.2 R6.10 Prob.16 T6.11 P1.9 L2.3 L6.4 16.9 17.1 P2.1 L2.4~ T2.6 P4.1 C7.2 C7.2 17.1 L7.30 Prob.18 C7.7ab L7.3b Prob.18 L7 .3~ Prob.18 L7.3d Prob.18 l7.6ab R7.9a €12.2.6 R7.4 T10.3.5ab R7.5 T10.3.5ab T7.6a T6.1 L7.3d C7.7a C7.8 T7.6b T6.5 L7.3d C7.76 T10.3.5ab C7.70 T6.1 L7.3a T7.6a C7.7b T6.5 L7.3a l7.6b T10.3.56 C7.8 T7.6a P12.2.2 R7.9a L7.3d R7.9b P2.1 R12.2.3 R7.9~ P4.1 P12.2.2 R7.9d

529



530 FLOWCHART

Ch8pt.r 2

Ll.1 P1.2d P1.56 L4.1 Pl.28 P1.2b Pl.48 P1.20 P1.2d L1.l P1.3 T2.13 R3.8.5a R4.1.4 T4.2.7 T5.1.2a P1.4. P1.4b Pl.4def P2.15 T4.2 C4.4 (24.5 P1.k P1.4ef L2.2 T2.13 P3.2 L4.1 P1.4d P1.4b L2.2 T2.13 P3.2 L4.1 R4.3 P4.1.5 T4.3.12 T4.4.26~ L5.2.4 P1.40 Pl.4bc P1.4f P1.41 P1.4bce P1.40 P1.2b L3.8.4 P1.5a P3.2 P3.4 L3.5 P3.6 T5.1 T4.6.1 T4.6.2 T4.6.3 C4.6.4 14.6.5 L4.10.6 T5.2.9

T5.3.7 15.3.11 P1.5b L1.l P2.15 P2.166 C5.3 c5.4 T4.3.8 C4.3.13 77.1.4ab T/.4.1 T8.3.1 T1.6 P2.la C2.17 P3.4 L3.5 P3.6 P6.2 l 7 . 1 . 4 T9.2.la P2.lb P2.9 T2.13 L7.2 L2.2 P1.44 L2.3 L2.5 T2.13 L2.3 L2.2 C2.4 C2.4 L2.3 P2.9 C2.11 R2.12 P2.16a L2.5 L2.2 C2.6 C2.6 L2.5 P2.9 C2.11 R2.12 L2.7 P2.9 L2.8 P2.9 T4.3.6 P2.9 P2.lb C2.4 C2.6 L2.7 L2.8 Prob.8 Prob.9 Prob.lOa C2.10 L4.1 R7.3 T4.3.6 C2.10 P2.9 Prob.lOa C2.11 R2.14 T12.4.1 C2.11 C2.4 C2.6 C2.10 Prob.9 T5.1 R5.2a A2.12 C2.4 C2.6 Prob.9 P3.4 L3.5 P3.6 T5.1 L4.6.5 T2.13 P1.3 P 1 . U P2.lb L2.2 Prob.lOa R2.14 P2.13 P2.166 P3.1 P3.2 P3.4 L95 P3.6 T4.2 (2.4 (2.5 T5.1 R5.36 (25.4 P4.2.9 T4.3.8 P4.3.9 P4.3.10 T4.3.12 C4.4.14 14.5.116 L4.5.13 T4.6.1 T4.6.2 T4.6.3 C4.6.4 L4.6.5 T4.10.1 L4.10.6 L5.2.4 T5.3.7 T6.1.3 T6.1.4

P2.15 T4.3.8 P4.3.9 P4.3.10 C4.3.13 L4.5.13 16.1.3 T6.1.4 T6.2.86 16.5.36 77.1.46 T7.4.1 T8.3.1 P2.15 P1.4b P1.5b T2.13 R2.14 P4.2.4 P2.16r C2.4 C2.17 P3.4 P3.6 77.1.4a T9.3.1 L9.4.1 P2.16b PlSb T2.13 m.17 T5.2.3 15.3.11 T9.2.la C2.17 P2.la P2.16ab P3.4 L3.5 P3.6 T10.4.5 P3.1 T2.13 P3.2 P 1 . U P1.5a T2.13 C3.3 L4.3.2 C3.3 P3.2 P3.4 P1.5a P2.la R2.12 T2.13 P2.16a C2.17 L3.5 Prob.8 Prob.lOa P6.1 T5.2.9 7 7 . 1 . 4 L3.5 P1.5a P2.1a R2.12 T2.13 C2.17 Prob.lOa P3.4 P3.6 P1.5a P2.la R2.12 T2.13 P2.16a C2.17 Prob.lOa L4.1 L1.l P 1 . U P2.9 T4.2 C4.4 C4.5 T4.2 P1.4b T2.13 L4.1 TA.4.2 P7.5 114.3 P1.4d C4.4 P1.4b T2.13 L4.1 TA.4.2 C4.5 P1.4b T2.13 L4.1 TA.4.2 T12.4.1 P4.8 TA.4.3 R4.7 R4.7 P4.6 T7.l T4.8.2 R4.8.36 C4.8.4 C4.8.5 C4.8.12 (24.8.13 15.1 P1.5a C2.11 R2.12 T2.13 Prob.15 PA.2.1 PA.2.4 C5.3 (25.4 P6.2 L7.2 T5.2.3 L5.2.4 R5.2a C2.11 R5.2b T2.13 R5.2c C5.3 P1.5b T5.1 C5.4 T2.13 T5.1 P6.2 T5.2.9 P6.1 P3.4 P6.2 T5.2.9 P6.2 P2.la T5.1 c5.4 P6.1 Prob.lOc T5.2.3 L5.2.4 l7.1 P1.2.1 C1.6.8 R4.7 P7.6 T4.8.2 R4.8.3a T4.8.10 L7.2 P2.lb T5.1 Prob.1Ob C7.4 R7.3 P2.9 C7.4 L7.2 P7.5 P1.5.2 T4.2 P7.6 R4.8.36 P7.6 l7 . l P7.5 P8.18 P8.lb P8.6 P8.2 P8.5c R8.3 P8.4 T8.7 P8Sr P8.5b P8.50 P8.2 P8.6 P8.1b P6.2.10 18.7 P8.4 T6.2.8a

~6.2.86 ~6.5.36 n.i.4ab n .4 . i ~8 .3 .1 ~9.1.6 ~ 1 0 . a ~10.2.10 ~2.14 c2.10 T2.13

Chapter 3

L1.l 11.2 L1.3 C1.6 L1.3 C1.5 T1.2 T1.7 T1.8 L1.4 C1.5 Cl.5 L1.4 L1.3 C1.6 T1.2 T1.7 C1.9 T1.7 L1.3 C1.6 Prob.3 722 T4.4.6 T4.6.3 11.8 L1.3 C1.9 V.8a 16.1.5 16.3.3 C1.9 C1.6 T1.8 T4.56 T6.3.4a T6.5.4 77.4.1 T9.2.16 T9.3.1 C10.2.6 C10.2.7 T10.4.6 T11.2.3 TA.1.2 L2.1 T2.2 T4.5ab C7.4 C8.10 C9.2 14.1.1 P4.4.7 TA.8.1 12.2 T1.7 L2.1 C2.3 T4.56 P4.66 T7.2 R7.3 L7.5 L4.5.3 T4.5.1lb L4.5.15 R4.9.4 T4.9.9 T4.9.10 T10.2.2 TA.1.2 TA.8.1 C2.3 T2.2 L4.9.13 P2.4 P4.66 T4.1.1 73.3.46 TA.1.2 T3.1 C3.2 C3.3 P4.4 T4.56 C8.10 C9.2 T10.26 P10.4 T4.5.11~ L4.5.17 M0.4.5 TA.8.1 C3.2 T3.1 (29.3 R9.1.5 C3.3

FLOWCHART 531

T3.1 P10.4 77.3.1 T7.3.3 T7.4.1 111.2.3 R3.4 L8.1 L4.1 P4.2 P4.2 L4.1 TA.4.3 T4.5. T4.4.6 T4.5.19. C7.2.8 L4.3 T4.5b P4.66 L4.8.1 P4.4 T3.1 14.Sr L2.1 P4.2 14.56 14.5b C1.9 L2.1 T2.2 T3.1 L4.3 T4.5a P4.68 P4.1.6 14.4.2s C4.4.3 L4.8.1 P4.6b T2.2 P2.4 L4.3 W.1 P5.2 P6.6 P7.1 7B.l.lb PS.2 L5.1 P6.5 P7.1 T7.8a 14.5.11~ P5.3 (25.5 T5.6 16.2b P6.5 CQ.2 110.1 L4.5.10 T4.5.19. R5.4 756 P6.5 C5.5 P5.3 tri.6 P5.3 R5.4 Prob.14 P7.1 17.2 R7.3 77.8s C9.2 T4.4.6 14.6.3 L6.1 T6.3 T7.2 17.5 L6.2. 1 6 . a 16.3 P6.5 C7.4 L6.2b P5.3 L6.2a L 6 . 2 ~ T6.3

L6.2e L6.2b 16.3 L6.1 L6.2ab Prob.16 77.2 T6.3.3 R6.4 Prob.16 R7.3 C9.2 P6.5 L5.1 P5.2 P5.3 Ft5.4 L6.2a P6.3.2 pT.1 L5.1 P5.2 T5.6 T7.86 U.4 .3 17.2 T2.2 T5.6 L6.1 T6.3 C7.4 T8.6 T9.1 R7.3 T2.2 T5.8 R6.4 C7.4 L2.1 L6.2a 17.2 16.1.5 P6.3.1 T6.3.46 C6.3.6 T6.5.4 793.1 L7.5 T2.2 L6.1 Prob.15 77.6 17.6 L7.5 L7.7 l7.8ab 14.5.1 14.8.10 77.8r T1.8 P5.2 T5.6 L7.7 T7.86 C8.10 C9.2 14.5.1

T4.8.10 l7.8b P7.1 L7.7 T7.8. TA.4.2 C9.3 C4.8.6 C4.8.15 L8.1 R3.4 P8.3 u1.2 P8.3 P8.3 L8.1 L8.2 T8.6 L8.4 P2.14 R8.5b T8.6 R8.51 P2.1.3 R8.Sb L8.4 18.6 18.6 Pmb.2.25 T7.2 P8.3 L8.4 R8.5b R8.7a C8.10 T9.1 73.4 77.1.4ab 17.4.1 19.1.4 R8.7r T8.6 19.4 R8.7b 18.8 C8.10 R8.91 R8.9b C8.10 L2.1 T3.1 T7.8a T8.6 T8.8 l9 . l T7.2 T8.6 Prob.13 C9.2 C9.3 R4.5.2 L4.5.17 C4.8.6 C4.8.15 14.9.17 77.1.4s TlO.4.1 C9.2 L2.1 T3.1 P5.3 T5.6 R6.4 T7.8a T9.1 C4.8.6 C9.3 C3.2 T7.8b T9.1 T4.2.S T4.2.6 T4.2.11 T4.2.12 T9.4 T8.6 R8.7a Prob.23 T4.2.5 T4.2.6 T4.2.11 T4.2.12 R4.5.2 L4.5.17 C4.8.6 C4.8.15 T4.9.17 T10.4.1 R9.5. R9.5b L10.1 P5.3 110.26 T10.2a P10.4 77.1.46 T9.1.4 111.4.1 110.2b T3.1

L10.1 P10.3 P10.4 T3.1 C3.3 TlO.2a

Chrptor 4

Tl.1 Prob.2.27 L3.2.1 P3.2.4 TA.9.1 P1.2 TA.4.2 P I S 75.19~ P1.3 12.7 R1.4 P2.1.3 P1.5 P2.1.4d P1.2 TA.4.2 P1.6 P3.4.6a 14.1 P1.7 P10.2.8 L2.1 T2.2 T12.4.1 T2.2 C1.2.8 T1.2.12 L2.1 78.1.4 T8.1.5 T8.3.1 L2.3 P2.4 T2.5 72.6 T2.11 T2.12 18.3.1 P2.4 P2.2.15 L2.3 12.5 T2.6 T2.11 T2.12 T2.5 T1.6.1 C3.9.3 T3.9.4 L2.3 P2.4 T2.7 T11.2.3 T2.6 T1.6.5 C3.9.3 T3.9.4 L2.3 P2.4 T5.1.2~ T2.7 P1.2.7 P2.1.3 P1.3 T2.5 C2.8 P2.9 T5.1.2 T8.3.1 T1O.l.l T12.2.4 T12.3.1 T12.4.1 C2.8 P1.2.1 T2.7 T8.3.1 P2.9 T2.2.13 T2.7 T5.1.2 T10.2.4 112.2.4 T12.3.1 T12.4.1 R2.10 P1.1.5~ T2.11 T1.6.1 C3.9.3 13.9.4 L2.3 P2.4 l2.12 T1.6.5 C3.9.3 T3.9.4 L2.3 P2.4 P3.1 T4.1 L3.2 P2.3.2 Prob.2.22 Prob.14 P3.3 P3.5 14.1 C4.4 15.6 15.18 T519a P9.2 19.3 T10.1 16.2.6 T6.2.8ab L9.4.1 P3.3 L3.2 L3.4 T10.3 L6.2.8b T9.4.3 T10.4.1 T12.2.4 112.3.1 T12.4.1 P3.5 L3.2 15.19a M.6 L2.2.8 P2.2.9 Pmb.3.7 C3.7 C3.7 T3.6 P5.3.5 183.3 110.4.1 13.8 P2.1.5b T2.2.13 R2.2.14 R5.5 T8.3.3 P3.9 P2.1.5b T2.2.13 R2.2.14 P5.3.5 P5.3.10 P3.10 P2.1.5b T2.2.13 R2.2.14 R3.11 T3.12 P2.1.4d T2.2.13 Prob.3.7 C3.13 T 5 . 1 1 ~ R6.1.6 T6.3.h C3.13 P2.1.5b R2.2.14 T3.12 14.1 T1.2.6 L1.4.2 C1.6.8 Pl.6 P3.1 L3.2 75.19~ 14.21 P3.4.6a P4.7 T10.1 110.3 19.4.3 110.4.1 T4.2b P2.1.4 Prob.2.llb 14.2~ P2.1.4d Pmb.2.1lb 75.11~ P9.19 79.14 C4.3 P3.4.6a P3.7.1 (2.4 T6.2 P9.19 C4.4 L3.2 (2.3 R4.5 Prob.22 14.6 T3.1.7 P3.4.2 T3.5.6 TA.10.5 P4.7 L3.2.1

T4.2a R4.8a R4.8b E4.9 Prob.23 L4.10 R4.12 T4.11 C4.13 R4.12 L4.10 C4.13 T4.11 C4.14 T2.2.13 (24.15 C4.15 C4.14 R4.16 U.1 L3.7.7 T3.7.8a

T5.4 15.17 79.17 R5.2 T3.9.1 T3.9.4 T5.4 W.3 T3.2.2 TA.lO.l T5.4 T10.4.1 75.4 L5.1 R5.2

532 FLOWCHART

L5.3 T5.3.10 R5.5 T3.6 W.6 L3.2 U.8 W.0 L5.10 U.10 P3.5.3 L5.9 T511b TS.118 L5.16 T5.11b TS.llb T2.2.13 T3.2.2 L6.10 T5.11a L6.17 7 5 1 1 ~ TS.110

T3.3.1 P3.5.2 T3.12 T4.2~ T5.11b R5.12 L5.13 T2.2.13 R2.2.14 €5.14 LS.15 Prob.2.26 T3.2.2 L5.16 T5.3.6 C7.5.3 U.16 L5.15 75.11~ T6.1 L6.5 T6.1.4 T6.2.86 LS.17 T3.3.1 T3.9.1 T3.9.4 L5.1 T5.11b LS.18 L3.2 7'5.19. TS.10a L1.2.3 P3.4.2 P3.5.3 L3.2 P3.5 L5.18 L5.20 75.19b 75.10~ T4.1 P1.2 T5.1Qd L5.20 LS.20 P3.5.3 L5.10 75.19.d T6.1 P2.1.5a M.2.13 L5.16 C6.4 77.4.1 T6.2 P2.1.58 T2.2.13 C4.3 Prob.27 T6.3 P2.1.5a T2.2.13 M.1.7 T3.5.6 C6.4 C6.4 P2.1.5a T2.2.13 T6.1 T6.3 L6.5 P2.1.5~ R2.2.12 T2.2.13 L5.16 T6.6 T6.6 L6.5 T7.1 P5.3.5 T5.3.10 L7.2 n.3 Prob.29 L8.1 L3.4.3 P3.4.6a T8.2 R6.3.5a T8.2 T1.6.9 R2.4.7 T2.7.1 L8.1 C8.4 C8.5 C8.6 R8.38 T2.7.1 R8.3b R2.4.7 P2.7.5 R8.3~ C8.7 C8.9 R8.3d C8.4 R2.4.7 T8.2 C8.S Fl2.4.7 T8.2 C8.6 T3.7.8b M.9.1 C3.9.2 T3.9.4 T6.2 C8.7 C8.9 T9.2.la T12.4.1 C8.7 P1.5.1 A8.3~ C8.6 112.2.4 T12.3.1 R8.88 T1.6.1 R8.W C8.9 R 8 . 3 ~ C8.6 T9.1.3 110.1.1 T10.3.5a T8.10 T2.7.1 L3.7.7 T3.7.8a C8.12 C8.13 C8.15 R8.11 C8.16 C8.17 a 1 2 R2.4.7 T8.10

C8.13 R2.4.7 T8.10 R8.14 C8.15 T3.7.6b T3.9.1 T3.9.4 T8.10 C8.16 C8.17 C8.16 P1.5.1 R8.11 C8.15 T9.4.3 P10.4.2 C8.17 R8.11 C8.15 T10.4.1 P8.18 L9.1 P9.2 P9.2 P1.1.5~ C1.1.6 C1.6.8 L3.2 L9.1 T0.3 L10.2.1 79.3 C1.6.8 L3.2 P9.2 T10.4.6 R9.4 T3.2.2 L0.S R9.6 U.7 M.2.2 CQ.8 TQ.9 T3.2.2 TQ.10 T3.2.2 R9.11 T9.12 T10.4.6 LQ.13 C3.2.3 TQ.14 R9.15 L9.16 T9.17 t9.17 T3.9.1 T3.9.4 LS.l L9.16 Prob.41 TA.8.1 P9.19 T10.4.6 P9.18 PO.19 T4.2~ C4.3 T9.17 T10.3 T IO. l T2.2.13 Prob.2.23 L3.2 L4.2a T10.3 P10.2 PA.4.5 110.3 T10.3 L3.4 T4.2a T1O.l P10.2 L10.5 L10.6 P9.19 R10.4 LIO.S T10.3 L10.6 P2.1.58 T2.2.13 TlO.3

Ch8pt.r 6

P1.l P1.3.3 Tl.2 T1.2~ T9.3.1 €10.43 T11.2.3 T1.2 T4.2.7 P4.2.9 Pl.1 P3.1 T9.3.1 T1.2a P2.1.3 Tl.2bc T11.2.3 T1.2b T1.2a C3.4 M.7 M.8 T3.11 €9.3.2 T1.2~ T1.6.5 T4.2.6 P1.l T1.2a L2.1 T2.3 L2.2 T2.3 T2.3 P2.2.16b T2.5.1 P2.6.2 L2.1 L2.2. L2.4 T2.6 T3.11 L2.4 P2.1.4d T2.2.13 T2.5.1 P2.6.2 T2.3 T2.6 L2.7 L2.5 Prob.11 U.8 T2.6 T2.3 L2.4 L2.7 L2.8 T2.9 T3.1 M.3 M.7 T7.1.1 L2.7 L2.4 T2.6 T2.9 L2.8 L2.5 T2.6 P3.1 73.3 T2.9 P2.1.5a P2.3.4 C2.5.4 P2.6.1 T2.6 L2.7 L2.11 Prob.12 T2.12 P3.1 C3.4 73.8 l7.1.1 17.1.2 17.4.1 €9.3.2 T10.4.5 R2.10 L2.11 T2.9 T2.12 T2.0 P3.1 T3.3 T6.5.36 P3.1 T1.2b T2.6 L2.8 T2.9 T2.12 L3.2 TA.10.1 T3.3 T3.3 T2.6 L2.8 T2.12 L3.2 C3.4 T6.5.3a C3.4 T1.2b T2.9 T3.3 T3.10 T8.2.3 T8.2.6 P3.5 C4.3.7 P4.3.9 T4.7.1 Prob.4.10 13.10 18.1.7 T8.2.3 T8.2.6 T3.6 L4.5.15 T8.2.6 T3.7 P2.1.5a P2.2.13 T1.2b T2.6 TA.S.1 73.11 T3.8 T1.2b T2.9 TA.5.1 T8.2.3 R3.9 T3.10 P4.3.9 T4.5.4 T4.7.1 Prob.4.19 C3.4 P3.5 T8.2.3 T8.2.6 T3.11 P2.1.5a P2.2.16b T1.2b T2.3 T3.7 T11.3.2

Chaptor 6

Tl.la Tl.lb Tl.lb L3.5.1 Tl.la T1.5 R1.2 Tl.3 T2.2.13 R2.2.14 T9.3.1 71.4 T2.2.13 R2.2.14 L4.6.16 Pmb.4.45 Prob.12 T1.6 T3.1.8 (23.7.4 T1.l.b 19.1.4 R1.Q T4.3.12 L2.1 TA.11.3 T2.2a TA.11.3 T2.26 T2.2b T2.2a T4.1b 15.1 R2.3 Prob.1 P2.4 TA.ll.3 R2.5 L2.6 L4.3.2 L2.7 12.8ab P2.10 L2.7 L2.6 T3.4a T5.4

T2.8r T2.8.7 L4.3.2 L2.6 T4.1b T5.1 T2.m Prob.1.23 T2.2.13 R2.2.14 Prob.2.24 L4.3.2 L4.3.4 L4.5.16 Prob.4.45 L2.6 R2.98 R2.9b P2.10

FLOWCHART 533

P2.8.6 L2.6 T4.1b T5.1 P3.1 (23.7.4 P3.2 P3.6.5 Prob.5 T3.3 T3.1.8 T3.6.3 T3.48 C3.1.9 T4.3.12 L2.7 T3.46 C3.6 T3.4b P3.2.4 C3.7.4 T3.4a Prob.7 R3.58 14.8.1 R3.5b C3.6 C3.7.4 T3.4a T4.18 T4.lb T2.2b T2.8a P2.10 T 4 . 1 ~ T4.10 T4.lb T11.2.1 15.1 T2.2b T2.8a P2.10 R5.2a T5.3a T11.3.1 R5.28 T5.1 R5.2b T5.38 T5.3.3 T5.1 T5.3b T2.2.13 R2.2.14 T5.2.12 T5.4 C3.1.9 C3.7.4 L2.7 R5.S

Chapter 7

T1.l T5.2.6 T5.2.9 T1.46 T1.2 T5.2.9 T1.48 R1.3 Prob.2 T1.48 P2.1.5b P2.2.la T2.2.13 R2.2.14 P2.2.16a P2.3.4 T3.8.6 T3.9.1 Prob.3.22~ T1.2 Prob.7 T3.1 T3.3 T1.4b P2.1.5b T2.2.13 R2.2.14 T3.8.6 T3.10.2a T1.l Prob.7 PA.2.2 PA.2.3 T9.3.1 R1.5 L2.1 P2.2 P2.6 P2.2 L2.1 R2.3 C2.4 C2.5 R2.3 P2.2 T3.1 C2.4 P2.2 M.1 C2.5 P2.2 P2.6 L2.1 C2.7 C2.8 T3.3 T12.4.1 C2.7 P2.6 73.3 C2.8 P3.4.2 P2.6 PA.4.5 T3.1 C3.3.3 T1.4a R2.3 C2.4 R3.20 R3.2b T3.3 C3.3.3 Ti.& P2.8 C2.7 R3.4 T4.1 P2.1.5b T2.2.13 R2.2.14 C3.1.9 T3.3.3 T3.8.6 T4.6.1 T5.2.9 C4.2 C4.2 T4.1 Prob.13 T5.1 C5.2 C5.3 C5.2 T5.1 Prob.17 C5.3 L4.5.15 T5.1 Prob.17 R5.4 C5.5 T11.3.1 C5.5 R5.4 T11.3.1 T5.6

Chapter 8

Tl.1 €12.33 C1.2 Prob.l R1.3 Prob.2 T1.4 T4.2.2 11.5 R1.3.2 T4.2.2 T1.6 P1.3.3 T1.7 P5.3.5 T2.1 P1.3.3 P1.3.7 L2.2 T9.1.3 L2.2 T2.1 T2.3 C5.3.4 P5.3.5 T5.3.8 T5.3.10 P2.4 Prob.4 T2.5 T2.I P1.3.7 P2.4 TA.5.1 T2.6 C5.3.4 P5.3.5 T5.3.6 T5.3.10 R2.7 T2.8 P1.3.7 L2.9 PA.7.1 T1O.l.l L10.2.1 T10.3.5ab L2.9 T2.8 T3.1 P1.1.5~ R1.3.2 P1.3.3 L1.3.6 T1.4.3 T1.6.1 P2.1.5b T2.2.13 R2.2.14 T4.2.2 L4.2.3 T4.2.7 C4.2.8 TA.5.1 C3.2 C3.2 M.1 T3.3 C4.3.7 C4.3.8 73.4 T3.4 P1.3.3 T3.3 T3.S Prob.8 T3.6 Prob.8

Chapter 9

Tl.1 PA.4.5 R1.2 T1.3 T1.6.5 C4.8.9 T8.2.1 TA.1.2 T1.4 T3.8.6 T3.10.2a Prob.3.26 T4.4.2~ T6.1.5 L1.6 R1.5 C3.3.2 L1.6 T2.2.13 11.4 T2.10 P2.2.la P2.2.16b C4.8.6 Prob.3 T2.1b T2.lb C3.1.9 T2.la T 2 . 1 ~ T2.lc T2.1b T3.1 T2.2.16a C3.1.9 C3.7.4 P5.1.1 T5.1.2 T6.1.3 T7.1.4b €3.2 E3.2 T5.1.2b T5.2.9 Prob.7.3 T3.1 L4.1 P2.2.16a L4.3.2 TA.5.1 T4.3 "4.2 P1.5.4 L4.3.4 T4.4.2a L4.1 Prob.7 T4.3 P1.5.4 L4.3.4 T4.4.2a C4.8.16 L4.1 R4.4

Tl.1 T1.6.5 T4.2.7 C4.8.9 T8.2.8 Prob.1 T2.2 L2.1 P4.9.2 T8.2.8 T2.2 R2.3 T2.2 T3.2.2 T1.l L2.1 R2.3 L2.1 T2.4 Prob.3.5 P4.2.9 L2.10 Prob.3 PA.2.2 C2.6 C2.7 R2.5 PA.2.3 C2.7 P2.8 C2.6 C3.1.9 T2.4 PA.2.3 C2.7 C3.1.9 T2.4 R2.5 P2.8 T2.2.13 P4.1.7 R2.5 R2t9 R2.9 P2.8 L2.10 T2.2.13 T2.4 L3.1 P1.3.3 T3.5ab R3.2 L3.3 T3.5ab R3.4 T3.Sr R1.7.4 R1.7.5 T1.7.6b (34.8.9 T8.2.8 L3.1 L3.3 R3.6a €3.8 €3.9 T3.Sb R1.7.4 R1.7.5 T1.7.6b C1.7.7b T8.2.8 L3.1 L3.3 R3.6a R3.60 T3.5ab R3.6b R3.78 €3.8 E3.9 R3.7b E3.8 T3.5~ R3.7a E3.9 T3.5a R3.7a T4.1 P1.1.5b C1.1.6 T3.9.1 T3.9.4 L4.3.4 C4.3.7 T4.4.2a L4.5.3 (24.8.17 TA.5.1 P4.2 E4.4 P4.2 C4.8.16 T4.1 €4.3 E4.4 T4.5 E4.3 P5.1.1 P4.2 E4.4 T4.1 P4.2 T4.5 f4.6 T4.5 C2.2.17 Prob.2.29 73.3.1 75.2.9 P4.2 E4.4

T4.6 14.6 C3.1.9 T4.9.3 T4.9.12 T4.9.17 E4.4 T4.5 Pmb.12 T4.7 T4.7 T4.6

Chrptor 11

"2.1 T6.4. l~ TA5.1. 72.3 T4.1 R2.2 T2.3 C3.1.9 C3.3.3 T4.2.5 P5.1.1 T5.1.2r T2.1 T4.1 T3.1 T6.5.1 R7.5.4 C7.5.5 TA.5.1 T3.2 T5.3.11 R3.3 14.1 T3.10.2a T2.1 T2.3 R4.2 Pmb.5

Chrptor 12

L2.1 P2.2 P1.3.3 C1.7.8 R1.7.k R2.3 72.4 R2.3 R1.7.9b P2.2 m.5 T2.4 P1.4.1 Prob.3.25 T4.2.7 P4.2.9 L4.3.4 C4.8.7 P2.2 €2.6 €2.7 R2.S R2.3 €2.6 E2.6 L1.7.3d T2.4 R2.5 E2.7 Probl.6a T2.4 T3.1 Pmb.3.25 T4.2.7 P4.2.9 L4.3.4 C4.8.7 €3.3 R3.2 E3.3 T8.1.1 T3.1 T4.1 L1.4.2 C2.2.10 C2.4.5 Ptob.3.25 L4.2.1 T4.2.7 P4.2.9 L4.3.4 C4.8.6 P7.2.6

Appondlxor

Pl.1 T1.2 11.2 C3.1.9 T3.2.2 P3.2.4 P1.l T9.1.3 P2.3 P2.1 72.5.1 P2.2 P2.3 P2.2 P2.1 T7.1.46 T10.2.4 P2.3 T1.2 P2.1 7'7.1.46 R10.2.5 C10.2.6 P2.4 T4.2 T2.5.1 R2.5 P3.1 P3.2 14.1 14.2 72.4.2 C2.4.4 C2.4.5 73.7.86 P4.1.2 P4.1.5 P2.4 T4.3 T4.3 T4.2 P2.4.6 P3.4.2 C4.4 P4.5 C4.4 T4.3 P4.5 T4.3 P4.10.2 C7.2.8 T9.1.1 T5.1 T5.3.7 75.3.8 T8.2.5 T8.3.1 L9.4.l T10.4.1 T11.2.1 T11.3.1 16.1 C6.3 R6.2 C6.3 T6.1 P7.1 T8.2.8 P7.2 P7.2 P7.1 m.1 L3.2.1 M.2.2 T3.3.1 Prob.3.27 T4.9.17 R8.2 To.1 T4.1.1 T10.1 L4.5.3 L5.3.2 C10.3 R10.2 C10.3 T1O.l R10.4 T1O.S T4.4.6 P1l.l P11.2 T11.3 L6.2.1 T6.2.2a P6.2.4

WILEY SERIES IN PROBABILITY AND STATISTICS ES'IARLISIIEI) BY WAIdTItK A. SHEWIiAHT AND SAMUEL s. WlLKS

Editors: David J. Balding, Noel A. C. Cressie, Nicholas I . Fisher, lain M. Johnstone, J . B. Kadane, Geert Molenberghs. Louise M. Ryan, David W Scott, Adrian I;. M. Smith, Jozqf L. Teugels Editors Emeriti: Vic Burnett, J. Stuart Hunter. David G. Kendall

The Wifey Series in Probabifity andsrarisrics is well established and authoritative. It covers niany topics o f current research interest in both pure and applied statistics and probability theory. Written by leading statisticians and institutions, the t i t les span both state-of-the-art developments in the field and classical methods.

Reflecting the wide range o f current research in statistics, the series encompasses applied, methodological and theoretical statistics, ranging from applications and new techniques made possible by advances in computerized practice to rigorous treatment o f theoretical approac hcs.

This series provides essential and invaluable reading for all statisticians, whether in aca- demia, industry, government, or research.

7 ABRAHAM and LEDOLTER . Statistical Methods for Forecasting AGRESTI . Analysis o f Ordinal Categorical Data AGRESTI . An Introduction to Categorical Data Analysis AGRESTI . Categorical Data Analysis. Second Edidilion ALTMAN, GILL, and McDONALD. Nuinerical Issues in Statistical Computing for thc

AMARATUNGA and CAIIRERA . Exploration and Analysis o f DNA Microarray and

ANDEL . Mathematics of Chancc ANDERSON . An Introduction to Multivariatc Statistical Analysis, Third Edilion

ANDERSON, AUQUIER, HAUCK, OAKES, VANDAELE:., and WEISIJERG '

ANDERSON and LOYNES . The Tcaching o f Practical Statistics ARMITAGE and D A V I D (editors) * Advances in Biometry ARNOLD, BALAKRISHNAN, and NAGARAJA 0 Records

Social Scientist

Protein Array Data

* ANDERSON . The Statistical Analysis o fT imc Series

Statistical Mcthods for Comparative Studies

* ARTHANARI and DODGE . Mathematical Programming in Statistics * BAILEY . The Elements o f Stochastic Processes with Applications to the Natural

Sciences BALAKRISHNAN and KOUTRAS . Runs and Scans with Applications BARNETT . Comparativc Statistical Inference, Third Edifion UARNETT and LEWIS . Outliers in Statistical Data, Third Edition BAKTOSZYNSKI and NIEWIADOMSKA-UIJGAJ . Probability and Statistical Inference BASILEVSKY . Statistical Factor Analysis and Related Mcthods: Theory and

BASU and RIGDON . Statistical Methods for the Reliability of Rcpairable Systcms BATES and WATTS * Nonlinear Regression Analysis and I ts Applications BECHHOFER, SANTNER, and GOLDSMAN * Design and Analysis o f Expcrimcnts for

BELSLEY Conditioning Diagnostics: Collinearity and Weak Data in Regression

Applications

Statistical Selection, Screening, and Multiple Comparisons

*Now available in a lower priced paperback edition in the Wiley Classics Library. ?Now available in a lower priced paperback edition in the Wiley- Interscience Paperback Scries.



t BELSLEY, KUH, and WELSCH . Regression Diagnostics: Identifying Influential

BENDAT and PIERSOL Random Data: Analysis and Measurement Procedures,

BERRY, CHALONER, and GEWEKE . Bayesian Analysis in Statistics and

BERNARD0 and SMITH . Bayesian Theory BHAT and MILLER * Elements o f Applied Stochastic Processes, Third Edition BHATTACHARYA and WAYMIRE . Stochastic Processes with Applications

Data and Sources o f Collinearity

Third Edition

Econometrics: Essays in Honor of Arnold Zellner

t BIEMER, GROVES, LYBERG, MATHIOWETZ, and SUDMAN . Measurement Errors in Surveys

BILLINGSLEY * Convergence of Probability Measures, Second Edirion BILLINGSLEY * Probability and Measure, Third Edition BIRKES and DODGE . Alternative Methods of Regression BLISCHKE AND MURTHY (editors) ‘ Case Studies in Reliability and Maintenance BLISCHKE AND MURTHY * Reliability: Modeling, Prediction, and Optimization BLOOMFIELD . Fourier Analysis of Time Series: An Introduction, Second Edition BOLLEN . Structural Equations with Latent Variables BOROVKOV . Ergodicity and Stability o f Stochastic Processes BOULEAU * Numerical Methods for Stochastic Processes BOX . Bayesian Inference in Statistical Analysis BOX . R. A. Fisher, the Li fe of a Scientist BOX and DRAPER . Empirical Model-Building and Response Surfaces

* BOX and DRAPER * Evolutionary Operation: A Statistical Method for Process Improvement

BOX, HUNTER, and HUNTER Statistics for Experimenters: Design, Innovation, and Discovery, Second Editon

BOX and LUCERO * Statistical Control by Monitoring and Feedback Adjustment BRANDIMARTE * Numerical Methods in Finance: A MATLAB-Based Introduction BROWN and HOLLANDER . Statistics: A Biomedical Introduction BRUNNER, DOMHOF, and LANGER . Nonparametric Analysis of Longitudinal Data in

BUCKLEW . Large Deviation Techniques in Decision, Simulation, and Estimation CAlROLl and DALANG . Sequential Stochastic Optimization CASTILLO, HADI, BALAKRISHNAN, and SARABIA . Extreme Value and Rclated

CHAN . Time Series: Applications to Finance CHARALAMBIDES . Conibinatorial Methods in Discrete Distributions CHATTERJEE and HAD1 . Sensitivity Analysis in Linear Regression CHATTERJEE and PRICE * Regression Analysis by Example, Third Edilinn CHERNICK . Bootstrap Methods: A Practitioner’s Guide CHERNICK and FRllS . Introductory Biostatistics for the Hcalth Sciences CHILES and DELFINER . Geostatistics: Modeling Spatial Unccrtainty CHOW and LIU . Design and Analysis of Clinical Trials: Concepts and Mcthodologics,

CLARKE and DISNEY * Probability and Random Processes: A First Course with

Factorial Experiments

Models with Applications in Enginecring and Science

Second Edition

Applications, Second Edition * COCHRAN and COX . Experimental Designs, Second Edirion

CONGDON . Applied Bayesian Modelling CONGDON * Bayesian Statistical Modelling CONOVER . Practical Nonparametric Statistics, Third Edilion COOK . Regression Graphics COOK and WEISBERG * Applied Regression Including Computing and Graphics

*Now available in a lower priced paperback edition in the Wiley Classics Library. tNow available in a lower priced paperback edition in the Wiley-lnterscience Paperback Series.

COOK and WEISBERG . An Introduction to Regression Graphics CORNELL * Experiments with Mixturcs, Designs, Models, and the Analysis o f Mixture

COVER and THOMAS . Elements of Inforniatlon Theory COX . A Handbook o f Introductory Statistical Methods

CRESSIE . Statistics for Spatial Data, Revised Edition CSORGO and HORVATH . Limit Theorems in Change Point Analysis DANIEL . Applications o f Statistics to Industrial Experimentation DANIEL * Biostatistics: A Foundation for Analysis in the Health Sciences, Eighth Edirion

Data, Third Edition

* COX . Planning of Experiments

* DANIEL . Fitting Equations to Data: Computer Analysis o f Multifactor Data, Second Edition

DASU and JOHNSON . Exploratory Data Mining and Data Cleaning D A V I D and NAGARAJA . Order Statistics, Third Editiorz

DEL CASTILLO . Statistical Process Adjustment for Quality Control DEMARIS . Regression with Social Data: Modeling Continuous and 1,imitcd Rcsponsc

DEMIDENKO * Mixed Models: Theory and Applications DENISON, HOLMES, MALLICK and SMITH * Bayesian Methods for Nonlinear

DETTE and STUDDEN . The Theory o f Canonical Moments with Applications in

DEY and MUKERJEE . Fractional Factorial Plans DILLON and GOLDSTEIN . Multivariate Analysis: Mcthods and Applications DODGE . Alternative Mcthods of Rcgrcssion

* DECROOT, FIENBERG, and KADANE * Statistics and thc Law

Variables

Classification and Regression

Statistics, Probability, and Analysis

* DODGE and ROMlG . Sampling Inspection -1 ablcs, Second Edidrlron * DOOB . Stochastic Processes

DOWDY, WEARDEN, and CHILKO . Statistics for Research, 7hird Edition DRAPER and SMITH . Applicd Regression Analysis, Third Edirion DRY DEN and MARDIA . Statistical Shape Analysis DUDEWICZ and MISHRA . Modern Mathematical Statistics DUNN and CLARK . Basic Statistics: A Primer for thc Biomedical Sciences,

DUPlJlS and ELLIS * A Weak Convergencc Approach to the Theory of Large Deviations

ENDERS * Applied Econometric Time Series ETHIER and KURTZ . Markov Processes: Characterization and Convergcnce EVANS, HASTINGS, and PEACOCK . Statistical Distributions, Third Edition FELLER . An Introduction to Probability Theory and I ts Applications, Volume I ,

FISHER and V A N BELLE . Biostatistics: A Methodology for the ticalth Scicnces FITZMAURICE, LAIRD, and WARE . Applied Longitudinal Analysis

FLEISS * Statistical Methods for Rates and Proportions, Third Ediliori

FlJLLER . Introduction to Statistical Time Series. Second Edirion FULLER . Measurement Error Models GALLANT . Nonlinear Statistical Models GEISSER Modes o f Parametric Statistical Inference GEWEKE . Contemporary Bayesian Economctrics and Statistics GHOSH, MUKHOI’ADHYAY, and SEN . Scquential Estimation

Third Edirion

* ELANDT-JOHNSON and JOHNSON . Survival Models and Data Analysis

Third Edirion, Revised; Volume II, Second Edirion

* FLEES. The Design and Analysis of Clinical Experiments

t FLEMING and HARRINGTON . Counting Processes and Survival Analysis

*Now available in a lower priced paperback edition in the Wiley Classics Library. ?Now available in a lower priced paperback edition in the Wiley-lntcrscicncc Paperback Scriei.

GIESBRECHT and GUMPERTZ * Planning, Construction, and Statistical Analysis of

GlFl . Nonlinear Multivariate Analysis GIVENS and HOETING . Computational Statistics GLASSERMAN and YAO . Monotone Structure in Discrete-Event Systems GNANADESIKAN . Methods for Statistical Data Analysis of Multivariate Observations,

GOLDSTEIN and LEWlS 3 Assessment: Problems, Development, and Statistical Issues GREENWOOD and NIKULIN * A Guide to Chi-Squared Testing GROSS and HARRIS * Fundamentals of Queueing Theory, Third Edition

Comparative Experiments

Second Edition

'f GROVES * Survey Errors and Survey Costs * HAHN and SHAPIRO * Statistical Models in Engineering

HAHN and MEEKER Statistical Intervals: A Guide for Practitioners HALD . A History o f Probability and Statistics and thcir Applications Before I750 HALD * A History of Mathematical Statistics from 1750 to 1930

1' HAMPEL . Robust Statistics: The Approach Based on Influence Functions HANNAN and DEISTLER . The Statistical Theory o f Linear Systems HEIBERGER . Computation for the Analysis of Designed Experiments HEDAYAT and SINHA - Design and Inference in Finite Population Sampling HELLER MACSYMA for Statisticians HINKELMANN and KEMPTHORNE * Design and Analysis o f Experiments, Volume I :

Introduction to Experimental Design HINKELMANN and KEMPTHORNE * Design and Analysis o f Experiments, Volume 2:

Advanced Experimental Design HOAGLIN, MOSTELLER, and TUKEY . Exploratory Approach to Analysis

of Variance HOAGLIN, MOSTELLER, and TUKEY * Exploring Data Tables, Trends and Shapes

* HOAGLIN, MOSTELLER, and TUKEY . Understanding Robust and Exploratory Data Analysis

HOCHBERG and TAMHANE . Multiple Comparison Procedures HOCKING . Methods and Applications o f Linear Models: Regression and the Analysis

HOEL * Introduction to Mathematical Statistics, FIfih Edilion HOGG and KLUGMAN . Loss Distributions HOLLANDER and WOLFE * Nonparametric Statistical Methods, Second Edition HOSMER and LEMESHOW . Applied Logistic Regression, Second Edition HOSMER and LEMESHOW . Applied Survival Analysis: Regression Modeling o f

HUBER . Robust Statistics HUBERTY * Applied Discriminant Analysis HUNT and KENNEDY . Financial Derivatives in Theoiy and Practice HIJSKOVA, BERAN, and DUPAC * Collected Works o f Jaroslav Hajek-

HUZURBAZAR . Flowgraph Models for Multistate Time-to-Event Data IMAN and CONOVER * A Modern Approach to Statistics

JOHN * Statistical Methods in Engineering and Quality Assurance JOHNSON . Multivariate Statistical Simulation JOHNSON and BALAKRISHNAN - Advances in the Theory and Practice of Statistics: A

JOHNSON and BHATTACHARYYA . Statistics: Principles and Methods, Fijih Edition JOHNSON and KOTZ . Distributions in Statistics

of Variance, Second Edition

Time to Event Data

with Commentary

t JACKSON . A User's Guide to Principle Components

Volume in Honor o f Samuel Kotz

*Now available in a lower priced paperback edition iii the Wiley Classics Library. ?Now available in a lower priced paperback edition in the Wiley-lnterscience Paperback Series.

JOHNSON and KOTZ (editors) . Leading Personalities in Statistical Sciences: From the

JOHNSON, KOTZ, and BALAKRISHNAN * Continuous Univariate Distributions,

JOHNSON, KOTZ, and BALAKRISHNAN . Continuous Univariate Distributions,

JOHNSON, KOTZ, and BALAKRISHNAN * Discrete Multivariate Distributions JOHNSON, KOTZ, and KEMP . Univariate Discrete Distributions, Second Edition JUDGE, GRIFFITHS, HILL, LUTKEPOHL, and LEE. The Theory and Practice of

JURECKOVA and SEN . Robust Statistical Procedures: Aymptotics and 1.nterrelations JUREK and MASON . Operator-Limit Distributions in Probability Theory KADANE . Bayesian Methods and Ethics in a Clinical Trial Design KADANE AND SCHUM . A Probabilistic Analysis of the Sacco and Vanzetti Evidence KALBFLEISCH and PRENTICE . The Statistical Analysis of Failure Time Data, Secotid

KASS and VOS . Geometrical Foundations o f Asymptotic Inference

, Seventeenth Century to the Present

Volume I , Second Edition

Volume 2, Second Edition

Econometrics, Second Edition

Edition

t KAUFMAN and ROUSSEEUW . Finding Groups in Data: An Introduction to Cluster Analysis

KEDEM and FOKIANOS . Regression Models for Time Series Analysis KENDALL, BARDEN, CARNE, and LE . Shape and Shape Theory KHURI . Advanced Calculus with Applications in Statistics, Second Edition KHURI, MATHEW, and SINHA . Statistical Tests for Mixed Linear Models

KLEIBER and KOTZ . Statistical Size Distributions in Economics and Actuarial Sciences KLUGMAN, PANJER, and WILLMOT. Loss Models: From Data to Decisions,

KLUGMAN, PANJER, and WILLMOT . Solutions Manual to Accompany Loss Models:

KOTZ, BALAKRISHNAN, and JOHNSON . Continuous Multivariate Distributions,

KOTZ and JOHNSON (editors) . Encyclopedia of Statistical Sciences: Volumes 1 to 9

KOTZ and JOHNSON (editors) . Encyclopedia of Statistical Sciences: Supplement

KOTZ, READ, and BANKS (editors) . Encyclopedia of Statistical Sciences: Update

KOTZ, READ, and BANKS (editors) . Encyclopedia o f Statistical Sciences: Update

KOVALENKO, KUZNETZOV, and PEGG . Mathematical Theory of Reliability of

LACHIN Biostatistical Methods: The Assessment of Relative Risks LAD . Operational Subjective Statistical Methods: A Mathematical, Philosophical, and

LAMPERTI * Probability: A Survey of the Mathematical Theory, Second Edition LANGE, RYAN, BILLARD, BRILLINGER, CONQUEST, and GREENHOUSE .

LARSON * Introduction to Probability Theory and Statistical Inference, Third Edition LAWLESS ' Statistical Models and Methods for Lifetime Data, Second Edifion LAWSON . Statistical Methods in Spatial Epidemiology LE . Applied Categorical Data Analysis LE . Applied Survival Analysis LEE and WANG . Statistical Methods for Survival Data Analysis, Third Edition

* KlSH . Statistical Design for Research

Second Edition

From Data to Decisions, Second Edition

Volume I , Second Edition

with Index

Volume

Volume I

Volume 2

Time-Dependent Systems with Practical Applications

Historical Introduction

Case Studies in Biometry

+Now available in a lower pnced paperback edition in the Wilcy Classics Library. ?Now available in a lower priced paperback cdition in thc Wiley-lntcrscience Paperback Series.

LEPAGE and BILLARD - Exploring the Limits o f Bootstrap LEYLAND and GOLDSTEIN (editors) * Multilevel Modelling of Health Statistics Ll A 0 * Statistical Group Comparison LINDVALL * Lectures on the Coupling Method LINHART and ZUCCHINI . Model Selection LITTLE and RUBIN Statistical Analysis with Missing Data, Second Edition LLOYD - The Statistical Analysis of Categorical Data LOWEN and TElCH * Fractal-Based Point Processes MAGNUS and NEUDECKER Matrix Differential Calculus with Applications in

MALLER and ZHOU Survival Analysis with Long Term Survivors MALLOWS * Design, Data, and Analysis by Some Friends o f Cuthbert Daniel MANN, SCHAFER, and SINGPURWALLA - Methods for Statistical Analysis o f

MANTON, WOODBURY, and TOLLEY . Statistical Applications Using Fuzzy Sets MARCHETTE * Random Graphs for Statistical Pattern Recognition MARDIA and JUPP Directional Statistics MASON, GUNST, and HESS * Statistical Design and Analysis of Experiments with

McCULLOCH and SEARLE Generalized, Linear, and Mixed Models McFADDEN Management o f Data in Clinical Trials

* McLACHLAN Discriminant Analysis and Statistical Pattern Recognition McLACHLAN, DO, and AMBROISE * Analyzing Microarray Gene Expression Data McLACHLAN and KRISHNAN * The EM Algorithm and Extensions McLACHLAN and PEEL . Finite Mixture Models McNElL * Epidemiological Research Methods MEEKER and ESCOBAR * Statistical Methods for Reliability Data MEERSCHAERT and SCHEFFLER * Limit Distributions for Sums of Independent

MICKEY, DUNN, and CLARK a Applied Statistics: Analysis of Variancc and

Statistics and Econometrics, Revised Edifion

Reliability and Life Data

Applications to Engineering and Science, Second Edition

Random Vectors: Heavy Tails in Theory and Practice

Regression, Third Edition * MILLER . Survival Analysis, Second Edition

MONTGOMERY, PECK, and VlNlNG . Introduction to Linear Regression Analysis,

MORGENTHALER and TUKEY . Configural Polysampling: A Route to Practical

MUIRHEAD * Aspects of Multivariate Statistical Theory MULLER and STOYAN - Comparison Methods for Stochastic Models and Risks MURRAY * X-STAT 2.0 Statistical Experimentation, Design Data Analysis, and

MURTHY, XIE, and JlANG * Weibull Models MYERS and MONTGOMERY * Response Surface Methodology: Process and Product

MYERS, MONTGOMERY, and VlNlNG - Generalized Linear Models. With

Third Edition

Robustness

Nonlinear Optimization

Optimization Using Designed Experimcnts, Second Edition

Applications in Engineering and the Sciences t NELSON * Accelerated Testing, Statistical Models, Test Plans, and Data Analyscs t NELSON * Applied Life Data Analysis

NEWMAN . Biostatistical Methods in Epidemiology OCHl * Applied Probability and Stochastic Processes in Engineering and Physical

OKABE, BOOTS, SUGIHARA, and CHlU * Spatial Tesselations: Concepts and

OLIVER and SMITH * Influence Diagrams, Belief Nets and Decision Analysis PALTA a Quantitative Methods in Population Health: Extensions o f Ordinary Regressions

Sciences

Applications of Voronoi Diagrams, Second Edirion

*Now available in a lower priced paperback edition in the Wiley Classics Library. ?Now available in a lower priced paperback edition in the Wiley-lnterscience Paperback Series.

PANKRATZ . Forecasting with Dynamic Regression Models PANKRATZ . Forecasting with Univariate Box-Jenkins Models: Concepts and Cases

PERA, TIAO, and TSAY . A Course in Time Series Analysis PIANTADOSI . Clinical Trials: A Methodologic Perspective PORT . Theoretical Probability for Applications POURAHMADI . Foundations o f Time Series Analysis and Prediction Theory PRESS . Bayesian Statistics: Principles, Modcls. and Applications PRESS . Subjective and Objective Baycsian Statistics, Second Edition PRESS and TANUR . Thc Subjectivity o f Scientists and the Daycsian Approach PIJKELSHEIM . Optimal Experimental Design PURI, VILAPL.ANA, and WERTZ . New Perspectives in Theoretical and Applicd

* PARZEN . Modern Probability Theory and Its Applicalions

Statistics '1 PUTERMAN . Markov Decision Processes: Discrete Stochastic Dynamic Programming

* RAO . Linear Statistical Inference and I t s Applications, Second Edirion QIU . Image Processing and Jump Regression Analysis

RAUSAND and HOYLAND . System Reliability Theory: Models, Statistical Methods,

RENCHER . Linear Models in Statistics RENCHER . Methods of Multivariate Analysis, Second Edrlion RENCHER . Multivariatc Statistical lnferencc with Applications

RIPLEY . Stochastic Simulation ROBINSON . Practical Strategies for Expcritnenting ROHATCI and SALEH . An Introduction to Probability and Statistics, Second Edition ROLSKI, SCHMIDI.1, SCtIMIDT, and TEIJCELS Stochastic I'rocesscs for Insurance

ROSENBERGER and I A C H I N . Randomization in Clinical Trials: Thcory and Practice ROSS . Introduction to Probability and Statistics for Engincers and Scientists

and Applications, Second Edirion

* RIPLEY . Spatial Statistics

and Finance

t ROUSSEEUW and LEROY . Robust Regression and Outlier Detection * RUBIN . Multiple Imputation for Nonresponse in Surveys

RUBINSTEIN . Simulation and the Monte Carlo Method RlJBlNSTElN and MELAMED . Modern Simulation and Modeling R Y A N . Modern Regression Methods RYAN . Statistical Methods for Quality Improvement, Second Edition SALTELLI, CHAN, and SCOTT (editors) Sensitivity Analysis

SCll lMEK . Smoothing and Regression: Approaches, Computation, and Application SCtiOTT . Matrix Analysis for Statistics, Second Edition SCHOUTENS . Levy Processes in Financc: Pricing Financial Derivatives SCHUSS . Theory and Applications o f Stochastic Diflerential Equations SCOTT . Multivariate Density Estimation: Theory, Practice. and Visualization

SEARLE . Linear Models for Unbalanced Data SEARLE . Matrix Algcbra Useful for Statistics SEARLE, CASELLA, and McCULLOCH . Variance Components SEARLE and WILLETT . Matrix Algebra for Applied Economic5 SEBER and LEE * Linear Regrcssion Analysis, Second Edrtion

* SCHEFFE . The Analysis of Variance

* SEARLE . Linear Models

f' SEBER . Multivariate Observations t SEBER and W I L D . Nonlinear Regression

SENNOTT- . Stochastic Dynamic Programniing and thc Control o f Queueing Systems * SERFLING . Approximation Theorems of Mathematical Statistics

SHAFI:K and VOVK . Probability and Financc: I t 's Only a Game!

* N o w available in a lower priced paperback edition in the Wiley Classics Library. ?Now available in a lowcr priced paperback cdition in thc Wiley -1ntersciencc Paperback Serics.

SILVAPULLE and SEN * Constrained Statistical Inference: Inequality, Order, and Shape

SMALL and McLEISH * Hilbert Space Methods in Probability and Statistical Inference SRIVASTAVA . Methods of Multivariate Statistics STAPLETON . Linear Statistical Models STAUDTE and SHEATHER. Robust Estimation and Testing STOYAN, KENDALL, and MECKE . Stochastic Geometry and Its Applications, Second

STOYAN and STOYAN * Fractals, Random Shapes and Point Fields: Methods of

STYAN . The Collectcd Papers ofT. W. Anderson: 1943-1985 SUTTON, ABRAMS, JONES, SHELDON, and SONG. Methods for Meta-Analysis in

TAN AKA . Time Series Analysis: Nonstationary and Noninvertible Distribution Theory THOMPSON . Empirical Model Building THOMPSON . Sampling, Second Edition THOMPSON . Simulation: A Modeler's Approach THOMPSON and SEBER * Adaptive Sampling THOMPSON, WILLIAMS, and FINDLAY . Models for Investors in Real World Markets TIAO, BISGAARD, HILL, PENA, and STIGLER (editors) . Box on Quality and

TIERNEY . LISP-STAT: An Object-Oriented Environment for Statistical Computing

TSAY . Analysis o f Financial Time Scries UPTON and FINGLETON . Spatial Data Analysis by Example, Volume 11:

VAN BELLE . Statistical Rules o f Thumb VAN BELLE, FISHER, HEAGERTY, and LUMLEY . Biostatistics: A Methodology for

the Health Sciences, Second Edition VESTRUP . The Theory o f Measures and Integration VlDAKOVlC . Statistical Modeling by Wavelets VINOD and REAGLE Preparing for the Worst: Incorporating Downside Risk in Stock

Market Investments WALLER and COTWAY Applied Spatial Statistics for Public Health Data WEERAHANDI . Generalized Inference in Repeated Measures: Exact Methods in

MANOVA and Mixed Models WEISBERG . Applied Linear Regression, Third Edition WELSH . Aspects of Statistical Inference WESTFALL and YOUNG . Resampling-Based Multiple Testing: Examples and

WHITTAKER * Graphical Models in Applied Multivariate Statistics WINKER . Optimization Heuristics in Economics: Applications of Threshold Accepting WONNACOTT and WONNACOTT * Econometrics, Second Edition WOODING . Planning Pharmaceutical Clinical Trials: Basic Statistical Principles WOODWORTH - Biostatistics: A Bayesian Introduction WOOLSON and CLARKE . Statistical Methods for the Analysis o f Bioincdical Data,

WU and HAMADA . Experiments: Planning, Analysis, and Parameter Design

YANG . The Construction Theory of Denumerable Markov Processes

ZHOU, OBUCHOWSKI, and McCLlSH . Statistical Methods in Diagnostic Medicine

Restrictions

Edidilion

Geometrical Statistics

Medical Research

Discovery: with Design, Control, and Robustness

and Dynamic Graphics

Categorical and Directional Data

Methods for p-Value Adjustment

Second Edilion

Optimization

* ZELLNER . An Introduction to Bayesian Inference in Econometrics

*Now available in a lower priced paperback edition in the Wiley Classics Library. ?Now available in a lower priced paperback edition in the Wiley-laterscience Paperback Series.

ethier s.n., kurtz t.g. markov processes characterization and convergence

Technology