gaussian random processes

Optimization
w. Hildenbrand
I.A. lbragimov Y.A. Rozanov
Springer-Verlag New York Heidelberg Berlin
lA. Ibragimov Lomi Fontanka 25 Leningrad 191011 U.S.S.R.
Editorial Board
A. V. Balakrishnan Systems Science Department University of Calif omia Los Angeles, California 90024 USA
Y.A. Rozanov V.A. Steklov Mathematics Institute Zazilov St. 42 Moscow 3-333 U.S.S.R.
W. Hildenbrand Institut fur Gesellschafts und Wirtschaftswissenschaften der Universitat Bonn 0-5300 Bonn Adenauerallee 24-26 German Federal Republic
AMS Subject Classifications: 6OGlO, 6OGl5, 6OG35
Library of Congress Cataloging in Publication Data
Ibragimov, II'dar Abdulovich. Gaussian random processes.
(Applications of mathematics; 9) Translation of Gaussovskie sluchainye protsessy. Bibliography: p. 1. Stochastic processes.
Antol'evich, joint author. QA274.4.I2613 519.2
I. Rozanov, Iurii II. Title.
78-6705
The original Russian edition GAUSSOVSKIE SLUCHAINYE PROTSESSY was published in 1970 by Nauka.
All rights reserved.
No part of this book may be translated or reproduced in any form without written permission from Springer-Verlag.
© 1978 by Springer-Verlag New York Inc. Softcover reprint of the hardcover 1st edition 1978
9 8 7 6 5 4 3 2 I
ISBN-13: 978-1-4612-6277-0 e-ISBN-13: 978-1-4612-6275-6 001: 10.1007/978-1-4612-6275-6
To Andrei Nickolajevich Kolmogorov
Preface
The book deals mainly with three problems involving Gaussian stationary processes. The first problem consists of clarifying the conditions for mutual absolute continuity (equivalence) of probability distributions of a "random process segment" and of finding effective formulas for densities of the equiva lent distributions. Our second problem is to describe the classes of spectral measures corresponding in some sense to regular stationary processes (in par ticular, satisfying the well-known "strong mixing condition") as well as to describe the subclasses associated with "mixing rate". The third problem involves estimation of an unknown mean value of a random process, this random process being stationary except for its mean, i.e., it is the problem of "distinguishing a signal from stationary noise". Furthermore, we give here auxiliary information (on distributions in Hilbert spaces, properties of sam ple functions, theorems on functions of a complex variable, etc.).
Since 1958 many mathematicians have studied the problem of equivalence of various infinite-dimensional Gaussian distributions (detailed and sys tematic presentation of the basic results can be found, for instance, in [23]). In this book we have considered Gaussian stationary processes and arrived, we believe, at rather definite solutions.
The second problem mentioned above is closely related with problems involving ergodic theory of Gaussian dynamic systems as well as prediction theory of stationary processes. From a probabilistic point of view, this prob lem involves the conditions for weak dependence of the "future" of the proc ess on its "past". The employment of these conditions has resulted in a fruit ful theory of limit theorems for weakly dependent variables (see, for instance, [14], [22]); the best known condition of this kind is obviously the so-called condition of "strong mixing". The problems arising in considering regularity conditions reduce in the case of Gaussian processes to a peculiar approxima-
vii
Preface
tion problem related to linear spectral theory. The book contains the results of investigations of this problem which helped solve it almost completely.
The problem of estimating the mean is perhaps the oldest and most widely known in mathematical statistics. There are two approaches to the solution of this problem: first, the best unbiased estimates can be constructed on the basis of the spectral density of stationary noise; otherwise the least squares method can be applied.
We suggest one common class of "pseudobest" estimates to include best unbiased estimates as well as classical least squares estimates. For these "pseudobest" estimates explicit expressions are given, consistency conditions are found, asymptotic formulas are derived for the error correlation matrix, and conditions for asymptotic effectiveness are defined. It should be men tioned that the results relevant to regularity conditions and the mean estima tion are formulated in spectral terms and can automatically be carried over (within the "linear theory") to arbitrary wide-sense stationary processes.
Each chapter has its own numbering of formulas, theorems, etc. For ex ample, formula (4.21) means formula 21 of Section 4 of the same chapter where the reference is made. For the convenience of the reader we provide references to textbooks or reference books. The references are listed at the end of the book.
viii
Contents
CHAPTER I Preliminaries 1
1.1 Gaussian Probability Distribution in a Euclidean Space 1.2 Gaussian Random Functions with Prescribed Probability M~~ 2
1.3 Lemmas on the Convergence of Gaussian Variables 5 1.4 Gaussian Variables in a Hilbert Space 7 1.5 Conditional Probability Distributions and Conditional
Expectations 13 1.6 Gaussian Stationary Processes and the Spectral Representation 16
CHAPTER II The Structures of the Spaces H(T) and LT(F)
11.1 Preliminaries 11.2 The Spaces L +(F) and L -(F) 11.3 The Construction of Spaces Lr(F) When T Is a Finite
Interval 11.4 The Projection of L +(F) on L -(F) 11.5 The Structure of the a-algebra of Events U( T)
CHAPTER III
Equivalent Gaussian Distributions and their Densities 63
111.1 Preliminaries 63 111.2 Some Conditions for Gaussian Measures to be Equivalent 74 111.3 General Conditions for Equivalence and Formulas for
Density of Equivalent Distributions 85 111.4 Further Investigation of Equivalence Conditions 90
ix
Contents
IV.3 Conditions for Information Regularity IV.4 Conditions for Absolute Regularity and Processes with
Discrete Time IV.5 Conditions for Absolute Regularity and Processes with
Continuous Time
CHAPTER V
Complete Regularity and Processes with Discrete Time 144
V.1 Definitions and Preliminary Constructions with Examples 144 V.2 The First Method of Study: Helson-Sarason's Theorem 147 V.3 The Second Method of Study: Local Conditions 153 V.4 Local Conditions (continued) 163 V.5 Corollaries to the Basic Theorems with Examples 177 V.6 Intensive Mixing 181
CHAPTER VI
Complete Regularity and Processes with Continuous Time 191
VI.1 Introduction 191 VI.2 The Investigation of a Particular Function y(T; /1) 195 VI.3 The Proof of the Basic Theorem on Necessity 200 VI.4 The Behavior of the Spectral Density on the Entire Line 207 VI.5 Sufficiency 212 VI.6 A Special Class of Stationary Processes 217
CHAPTER VII
Filtering and Estimation of the Mean
VII.1 Unbiased Estimates VII.2 Estimation of the Mean Value and the Method of
Least Squares VII.3 Consistent Pseudo-Best Estimates VII.4 Estimation of Regression Coefficients
References
x
224
224
1.1 Gaussian Probability Distributions in a Euclidean Space
A probability distribution P in an n-dimensional vector space [Rn is said to be Gaussian if the characteristic function
<p(u) = flR n ei(u, X)P(dx), U E ~n
(here (u, x) = LUkXk denotes the scalar product of vectors u = (Ub ... , un) and x = (Xb ... , xn)) has the form
U E [Rn, (1.1)
where a = (ab ... , an) E [Rn is the mean and B is a linear self-adjoint non negative definite operator called a correlation operator; the matrix {Bd de fining B is said to be a correlation matrix. In this case
(U, a) = r (u, x)P(dx), JlRn
(Bu, v) = flR n [(u, x) - (u, a)] [(v, x) - (v, a)]P(dx), (1.2)
u, V E [Rn.
The distribution P with mean value a and correlation operator B is con centrated in an m-dimensional hyperplane L of [Rn (m being the rank of the correlation matrix), which can be expressed as
L = a + B[Rn
(L being the totality of all vectors Y E [Rn of the form y = a + Bx, x E [Rn).
1
I Preliminaries
In fact,
the distribution P being absolutely continuous with respect to Lebesgue measure dy in the hyperplane L, so that
P(F) = r p(y)dy, JrnL
(1.3)
where the distribution density p(y), y E L, has the form
1 { 1 -1 } P(Y)=(2nrI2detBexp -"2(B (y-a),(y-a)). (1.4)
Here det B denotes the determinant of the matrix that prescribes the operator B in the subspace [Rm = B[Rn, and B- 1 is the inverse on this subspace.
1.2 Gaussian Random Functions with Prescribed Probability Measure
Let (Q, W, P) be a probability space, i.e., a measurable space of elements W E Q with probability measure P on a a-algebra W of sets A £; Q.
Any real measurable function ~ = ~(w) on a space Q is said to be a random variable. The totality of random variables ~(t) = ~(w, t) (parameter t runs through a set T) is said to be a random function of parameter t E T. The random variables ~(t) themselves are said to be values of this random func tion ~ = ~(t); for fixed WE Q the real function ~(w, .) = ~(w, t) of t E T is said to be a sample function or a trajectory of the random function ~ = ~(t).
We shall consider another space X of real functions x = x(t) of t E T, which includes all trajectories ~ = ~(w, t), t E T, of the random function ~ = ~(t). (For instance, the space X = [RT of all real functions x = x(t), t E T, possesses this property.) Denote by !S the minimal a-algebra of sets of X containing all cylinder sets of this space, i.e., sets of the form
(2.1)
(the set indicated by (2.1) consists of the functions x = x(t) for which the values [x(t 1), ... , x(tn)] at the points t b ... , tn E T prescribe a vector that belongs to a Borel set r in an n-dimensional vector space [Rn). The mapping ~ = ~(w) under which each WE Q corresponds to a pertinent sample func tion ~(w, .) = ~(w, t) of t E T -an element of the space X-is a measurable mapping in a probability space (Q, W, P) onto a measurable space (X, !S). The sets A E m of the form A = {~ E B}-the preimages of sets B E !S under the mapping ~ = ~(w, .) indicated-form (in the aggregate) a a-algebra. This a-algebra m~ is minimal among a-algebras of the sets containing all sets of the form
(2.2)
2
1.2 Gaussian Random Functions with Prescribed Probability Measure
(the set indicated consists of the elements W E Q for which the values [~(w, t1), ..• , ~(w, tn)] prescribe a vector belonging to a Borel set r of an n-dimensional vector space []~n, or, in other words, the O"-algebra m:~ is to be generated by values ~(t), t E T. Probability measure P~ defined on a O"-algebra ~ by the relation
P~(B) = Pg E B}, BE~, (2.3)
is said to be a probability distribution of the random function ~ = ~(t) (on the pertinent function space X).
We shall discuss next the question: When is the family of real variables ~(t) = ~(w, t) given on a space Q (parameter t runs through a set T) a random function with the given probability distribution P~? More precisely, when does there exist probability measure P in the space Q related with the given distribution P~ by means of (2.3)? We assume in this case that the set ~(Q) of all sample functions ~(w, .) = ~(w, t) of t E T belongs to the space X.
It is readily seen that such a probability measure P exists if and only if the set ~(Q) has a complete exterior measure, i.e.,
P~(B) = 1 for B 2 ~(Q) (2.4)
for any measurable set B E X. In fact, if P~ is the probability distribution of the random function ~ =
~(t), for any set BE X in the complement of the set ~(Q) the set {~ E B} is empty and
P~(B) = Pg E B} = 0.
On the other hand, for any sets Bb B2 E ~,such that {~E Bd = {~E B 2}, the symmetric difference B1 0 B2 = (B1\B2) u (B2\B1) is contained in the complement of the set ~(Q), and, under the condition (2.4), P~(B1 0 B2) = 0, P~(B1) = P~(B2)' Hence the relation
Pg E B} = P~(B), BE~, (2.5)
defines the single-valued function P = peA) on the O"-algebra m:~ of all sets of the form A = {~ E B}, B E ~, generated by ~(t), t E T. Obviously, P is a probability measure and ~ = ~(t) is a random function on a probability space (Q, m:, P) with the given probability distribution P~.
The measure P on the O"-algebra m:~ generated by the variables ~(t), t E T, can be defined uniquely by finite-dimensional distributions PI! ..... In of which each is a Borel measure on IRn defined by
(2.6)
PI! ..... In being the probability distribution of the random vector [~(td, ... , Wn)]. In fact,
peA) = inf I peAk), (2.7) k
3
I Preliminaries
where the lower bound is taken over all sets Ak of the form (2.2), whose union covers the set A E~. In particular, this fact refers to the probability distribution P~ on the corresponding function space X ~it is the probability measure on the a-algebra '8 generated by the given variables ~(t) = ~(x, t) on the space X, i.e., the variables of the form
~(t, x) = x(t), XEX (2.8)
(where the parameter t, fixed for each functional ~(x, t) = x(t) of x E X, runs through the set T).
Denote by r x IRn - m the Borel set in an n-dimensional space of vectors [x(t!), ... , x(tn)] such that [x(ti,), ... , x(tiJ] E r (r is a Borel set in an m dimensional subspace IRm s; IRn) with the remaining coordinates x(tJ arbi trary. Finite-dimensional distributions are "compatible" in the sense that
ptl •... , tjr X IRn-m) = Ptil "'" tiJn (2.9)
for all sets of the above type. Let X = IRT be the space of all real functions x = x(t), t E T. According
to a well-known theorem due to Kolmogorov, * any given family of distri butions p tIo ... , tn prescribes a continuous additive function P (defined by (2.6), where the variables have the explicit form (2.8)), on the algebra of all cylinder sets (2.1). This function extends uniquely to a probability measure P on the a-algebra '8. The random function ~ = ~(t) with values ~(t) =
~(x, t) in the probability space (X, '8, P) has finite-dimensional distributions coinciding with the initial compatible distributions Ptl , ... ,tn'
Starting from probability distribution P = P¢ on the function space X, under the condition (2.4) we can define (see (2.5)) a probability measure on the corresponding space Q.
The random functions ~ = W) and ~ = ~(t) with values in the same space are said to be equivalent if with probability one (for almost all WE Q)
~(w, t) = ~(W, t)
for each fixed t E T. Obviously, the finite-dimensional distributions of equiv alent random functions coincide. Taking an equivalent random function ~ = ~(t) with the trajectories in any function space X, we can define (see (2.3)) a probability measure in this space as well.
Random variables are said to be Gaussian if their finite-dimensional dis tributions are Gaussian. More precisely, (when we deal with a random function ~ = ~(t) with parameter t E T under some parametrization), the values ~(t) = ~(w, t) and the function ~ = ~(t) itself are said to be Gaussian if all finite-dimensional distributions Ptl , ... , tn are Gaussian. Probability measure P on a a-algebra ~~ generated by all ~(t) is also said to be Gaussian.
Each of the finite-dimensional distributions Ptl , ... , tn of the Gaussian random function ~ = ~(t) has mean value [a(t!), ... , a(tn)] and correlation
* See [10], p. 150.
I.3 Lemmas on the Convergence of Gaussian Variables
matrix {B(tk' t j )} where a(t), t E T, is the mean value of the function ~ = ~(t), and B(s, t), s, t E T, is its correlation function:*
a(t) = M~(t),
B(s, t) = M[~(s) - a(s)][~(t) - a(t)], s, t E T. (2.10)
Therefore, the Gaussian measure P on a a-algebra ~~ can be defined uniquely by means of its mean value a(t), t E T, and its correlation function B(s, t), s, t E T.
The mean value a(t), t E T, can be arbitrary, and the correlation function B(s, t), s, t E T, need only satisfy the positive definiteness condition
n
for any tb ... , tn E T and real Cb ... , Cn'
(2.11)
For any function a(t), t E T, and a positive definite correlation function B(s, t), s, t E T, there exists a Gaussian random function with the mean a(t), t E T, and a correlation function B(s, t), s, t E T. Actually, Gaussian distri butions Pt " ... , tn with the mean [a(tl), ... , a(tn)] and correlation matrices {B(tb t j)} are compatible distributions, and define a Gaussian measure P in the space X = \RT of all real functions x = x(t) of t E T, on the a-algebra m = ~~, which can be generated by the given values ~(t) = ~(x, t) on X of the form (2.8) (parameter t runs through the set T).
I.3 Lemmas on the Convergence of Gaussian Variables
Let ~n = ~n(w), n = 1,2, ; .. , be a sequence of nindom variables on a prob ability space (Q, ~,P). The sequence ~n' n = 1, 2, ... , is said to be conver gent in probability on a set A E ~ to some variable ~ = ~(w) if for any e > 0
lim p({I~n - ~I > e} n A) = O. (3.1) n-+ DO
Let us recall that a sequence ~n' n = 1,2, ... , converges in probability if and only if this sequence is Cauchy, t i.e., on the same set A the sequence .1 nm = ~n - ~m' n, m = 1, 2, ... , converges to zero in probability.
Lemma 1. If a sequence of Gaussian variables ~"' n = 1, 2, ... , converges in probability on a set A E ~ of positive measure (P(A) > 0), it is convergent in the mean:
lim M[~n - ~y = O. (3.2) n-+ DO
* M~ denotes the expectation of a random variable ~ = ~(Q)) on a probability space (Q, W, P):
M~ = fn ~(w)P(dQ)). t See, for example, [10], p. 90.
5
I Preliminaries
Proof. We shall consider Gaussian variables Ll nm = ~n - ~m' For any 8 > 0
P{ILlnml > 8} = 2 fXJ;' exp { 2n(Jnm
where anm = MLl nm, (J;m = P(Llnm - anm)2. Suppose that the sequence ~m n = 1, 2, ... , is not convergent in the mean; this is equivalent to
lim (a;m + (J;m) > O. n, m- 00
It can be easily seen that under this condition for a positive 8 we have
lim P{ ILlnml > 8} ~ 1 - p/2, n, m-+ 00
where p = P(A) > O. But then
lim P( {ILlnml > 8} n A) ~ p/2, n, m- 00
which fact contradicts (3.1). Hence
lim MLl;m = lim (a;m + (J;m) = 0, n, m-+ 00 n, m-+ oc
i.e., the sequence ~n' n == 1, 2, ... , is Cauchy (a fundamental) in the mean and, therefore, is convergent in the mean.
In particular, if a sequence of Gaussian variables ~n' n = 1, 2, ... , is convergent with positive probability (i.e., convergent for all w from a set A E 'll of positive measure), it is convergent in the mean. D
Let us consider a sequence of independent Gaussian variables ~n' n = 1,2, ....
Lemma 2. The series I:= 1 ~; is convergent with positive probability if and only if the series I:=1 M~; is convergent.
Proof. Obviously, 00 00
and hence the convergence of the series I:= 1 M~; implies that the variable ~2 = I:= 1 ~;(w) is finite for almost all WE Q, i.e., the series I.% 1 ~; is con vergent with probability one. Let the series I:= 1 ~; be convergent with positive probability (by the well-known zero-one law* this series 'is con vergent with probability one as well). Then the sequence ~n' n = 1, 2, ... , converges to 0 in the mean: M~; ~ 0 for n ~ CX) (see Lemma 1). Let an =
* See, for example, [10], p. 157.
6
1.4 Gaussian Variables in a Hilbert Space
M~n' a; = M(~n - anf. Then
a2 + a 2 = M()")2 + i x 2 _1_ exp{ n n 'on Jlxl> 1 J2iian
where the random variables ~~ = ~~(w) are defined as
~~(w) = {o~n(W) for I~nl !( 1 for I~nl > 1.
For a; + a; -40 we have
so that M(~~)2 ~ a; + a;.
By the well-known three-series theorem* a necessary condition for the series 2:;;,,= 1 ~; of independent variables ~;, = 1,2, ... , to be convergent is that 2:;;,,= 1 M(~~)2 < 00. But M(~~)2 ~ a; + a;, and, consequently, it follows from the convergence of the series 2:;;,,= 1 ~; that 2:;;,,= 1 (a; + a;) < 00. 0
A random variable ~ in a Euclidean n-dimensional space IRn is said to be Gaussian if its probability distribution is Gaussian.
The random variable ~ E [Rn is Gaussian if and only if the real variable ~(u) = (u, ~) (equal to the scalar product ofthe elements u, ~ E [Rn) is Gaussian for each u E [Rn.
In fact, the value at a point u E [Rn of the characteristic function cp(u) of the random variable ~ E [Rn coincides with the value of the characteristic function of the real random variable ~(u) = (u, ~) at the point 1 and has the form
cp(u) = Mei(u,~) = eXP{i(U, a) - ~ (Bu, U)}. U E IRn
(see (Ll), where (u, a) is the mean and (Bu, u) is the variance of the Gaussian variable ~(u) = (u, ~)).
It is clear that the random variable ~ E [Rn is Gaussian if and only if the random function of the form ~(u) = (u, ~) of u E [Rn is Gaussian.
Let U be a complete separable Hilbert space and let ~ = ~(w) be a func tion on a probability space (Q, m:, P) with the values in U. The random element ~ of a Hilbert space U is said to be a random variable in U if the scalar product (u, ~) for each u E U is a real random variable, i.e., it is a measurable function on the probability space (Q, m:, P).
* See, for example, [10J, p. 166.
7
I Preliminaries
The random variable ~ in a Hilbert space U is said to be Gaussian if the real random variable ~(u) = (u, ~) is Gaussian for each u E U. This fact is equivalent to the fact that the random function ~(u) = (u, ~) of u E U is Gaussian since values ~(u) = (u, ~) as well as any vector values [~(Ul)' ... , ~(un)] are Gaussian.
In fact, for any vector A = [Al' ... , An] in [Rn the scalar product Lk= 1 Ak~(ud is equal to
Jl Ak~(Uk) = Ctl AkUb ~) = (u, ~), where u = Lk= 1 AkUk E U; by hypothesis, the variable ~(u) = (u, ~) is Gaussian.
Obviously, the mean
a(u) = M(u, ~), UE U,
of the random function ~(u) = (u, ~), U E U, is a linear functional, and the correlation function
B(u, v) = M[(u, ~) - a(u)][(v, ~) - a(v)], u, V E U,
is a bilinear positive functional on the Hilbert space U. In this case, since the scalar product (u, ~) is a continuous function of u E U for each fixed WE Q, a Gaussian function ~(u) = (u, ~) of u E U must be continuous in the mean (see Lemma 1):
lim M[(u, ~) - (v, ~)]2 = 0 (4.1) Ilu - vll-+ 0
(Ilull denotes the norm of the element u E U). But
M[(u, ~) - (v, m2 = a(u - V)2 + B(u - v, u - v)
and (4.1) implies that the functionals a(u) and B(u, v) are continuous. Being a linear continuous functional, the mean a(u) can be expressed as
a(u) = (u, a), UE U, (4.2)
for some element a in U. Any element a E U having the property that
(u, a) = In (u, ~(w))P(dw) (4.3)
for all u E U is said to be the mean* of a random variable ~ E U. Being a continuous positive bilinear functional, the correlation function B(u, v) can be expressed
B(u, v) = (Bu, v), U, VE U, (4.4)
where B is a linear positive (i.e., nonnegative self-adjoint) operator in a Hilbert space U called a correlation operator.
Let us show that the correlation operator B is completely continuous.
* For the integrability of functions with values in a Hilbert space, see, for example, [12], p. 59.
8
In fact, any orthonormalized sequence Vb V2, ... goes to zero weakly, so that the Gaussian variables ~n = (vn' ~), n = 1,2, ... , where ~ = ~(w) E U, goes to zero as n -4 00 for all WE Q. Therefore (see Lemma 1), they are convergent in the mean, i.e.,
M~; = (Bvn' Vn) -4 0
(here and further on we assume for simplicity of notation that the mean a E U is 0). If the operator B was not assumed completely continuous, outside some e-neighborhood of zero there would be an infinite number of spectral points (taking into account the multiplicity), and, therefore, an infinite number of invariant orthogonal subspaces for each element of which
(Bu, u) = r U(Elu, u);:' ellul12, Jlll>e
where B = SA dEl is the spectral representation of the continuous self adjoint operator B.
Further, we shall choose a complete orthonormalized basis of eigen elements Vb V2, ... of this completely continuous symmetric positive opera tor B corresponding to eigenvalues ai, a~, ... , The corresponding variables ~k = (Vb ~), k = 1,2, ... , are uncorrelated:
In this case 00 00
for j = k,
for j # k.
L ~r(w) = L (Vb ~(W))2 = 11~«(())112. 1 1
As is well known, uncorrelated Gaussian variables are independent and the convergence of the series Lr' ~r(w) (for all w) implies the convergence of the series Lr' Ma (see Lemma 2). Consequently,
00 00 00
L (BVb Vk) = L M~r = L ar < 00, 1 1 1
i.e., the correlation operator B is a nuclear operator:* for any orthonormal system Ub U2, .•. , E U,
(4.5)
Therefore, if we have a Gaussian random variable ~ E U, the random function ~(u) = (u, ~) of parameter u E U has a mean of the form (4.2) and a correlation function of the form (4.4) where the correlation operator B is a nuclear operator on the Hilbert space U.
Next, let ~(u), u E U, be an arbitrary Gaussian random function with a mean of the form (4.2) and a correlation function of the form (4.4), where B
9
I Preliminaries
is a nuclear operator on the Hilbert space U. Then there exists an equivalent random function ~(u), u E U, and a Gaussian random variable ~ = ~(w) in U such that
~(u) = (u, ~), UE U. (4.6)
The variable ~ E U indicated can be defined for almost all elementary outcomes w by the formula
00
~(w) = L ~(VdVb (4.7) k=l
where Vb V Z, ... is the complete orthonormal system of eigenelements of the nuclear operator B, and, by virtue of the relation
00 00
M L ~(Vk)Z = L B(Vb Vk) < 00 k=l k=l
for independent Gaussian variables ~(vd, ~(vz), ... , the series Lk'= 1 ~(Vk)2 is convergent with probability one. In fact, ~(u), u E U, is a random linear functional in the sense that with probability one
~(A1Ul + A2U2) = Al~(Ul) + )~2~(U2)
for any real Ab A2 and any elements Ub U2 E U since, as we can easily verify,
M[~(A1Ul + A2U2) - Al~(Ul) - A2~(U2)J2 = O.
Furthermore, the random functional ~(u) is continuous in the mean (see (4.1) and below), and, since
n
we have
~(U) = !~~ ~ Ct (u, V)Vk) = !~~ ktl (u, Vk)~(Vk) (in the sense of convergence in the mean); at the same time with probability one
(u,~) = !~~ Jl (U, Jl ~(Vk)Vk) = !~~ ktl (u, Vk)~(Vk)' so that we have the equality (4.6) with probability one for each value ~(u) of the primary random function ~(u), U E U.
Thus, we have arrived at the following result.*
Theorem 1. The Gaussian functional ~(u), U E U, on a Hilbert space U can be represented by (4.6) if and only if the mean a(u), U E U, is a continuous
* A survey of results related to distributions in linear spaces can be found, for example, in Yu. V. Prokhorov, 'The method of characteristic functionals," Proceedings of the 4th Berkeley Symposium, Vol. 2,1961, pp. 403-419.
10
linear functional, and the correlation function B(u, v), u, V E U, is a con tinuous bilinear functional, the corresponding operator B being a nuclear operator in the representation (4.4) of the bilinear functional B(u, v).
As is well known, a Hilbert space U can be identified with the adjoint space X of all linear continuous functionals on U. In fact, each functional x E X can be defined uniquely by the formula x = (u, x), U E U, where x E U is a fixed element of the space U. Following the exposition in Section 2, any Gaussian random function ~(u), U E U, of the form (4.6) corresponds to a probability distribution p~ in a Hilbert space X = U (sample functionals ~(u) = (u, ~(w)), U E U, belong to X). The Gaussian measure p~ is defined on the rr-algebra 5B generated by cylinder sets ofthe space X = U of the form
[(Ul' x), ... , (un, x)] E r, (4.8)
where U b ... , Un E U and r are Borel sets in an n-dimensional Euclidean space; the rr-algebra 5B is generated, obviously, by variables of the form
~(x, u) = (u, x), XEX, (4.9)
where the parameter u runs through the space U = X. For any element a E U and a positive nuclear operator B there exists a
Gaussian random function ~(u), u E U, of the form (4.6) with a mean a E U and a correlation operator B. There also exists a Gaussian function of this type with values defined by (4.9) on the probability space (X, 5B, P~).
Example. Gaussian Variables in the Function Space ;t?2(T). Let ~ = ~(t) be a Gaussian random process on an interval T of the real line with mean a(t), t E T, and let correlation function B(s, t), s, t E T, satisfy the following con dition: for all s, t E T,
lim [a(s) - a(t)] = 0, s-->t
(4.10) lim [B(s, s) - 2B(s, t) + B(t, t)] = 0. s-->t
This condition implies that the random process ~ = ~(t) is continuous in the mean:
lim M[~(s) - ~(t)]2 = 0. s-->t
As is well known, * in this case there exists an equivalent measurable process (with values ~(t) = ~(w, t)) such that the function ~ = ~(w, t) with respect to a pair of variables (w, t) on the product of spaces Q x T is measurable. We shall consider that the initial Gaussian process ~ = ~(t) itself is Gaussian. Assume that the condition
ST B(t, t) dt < 00, (4.11)
11
By Fubini's theorem on iterated integration
IT M~2(t)dt = In IT e(w, t)dtP(dw) < 00
and almost all sample functions ~(w, .) = ~(w, t) of t E T belong to a Hilbert space 22(T) of real square-integrable functions u = u(t) of t E T with the scalar product
(u, v) = IT u(t)v(t)dt.
We redefine the values ~(w, t) for those w E Q for which the sample func tions ~(w, .) = ~(w, t), t E T, do not appear in 22 (the set of such WE Q has measure 0) and go over to a measurable Gaussian process ~ = ~(t) whose sample functions belong to the Hilbert space 22. This random function ~ = ~(w, .) can be treated as a random element in the Hilbert space 22.
Consider the scalar products
(u, ~) = IT u(t)~(w, t)dt, UE u.
Since ~ = ~(w, t) is a measurable function of the variables (w, t), for each fixed u E U the real function (u, ~) of WE Q is also measurable, i.e., is a random variable. Therefore, ~ = ~(w, .) is a random variable in a Hilbert space 2 2(T). By Fubini's theorem the random function ~(u) = (u, ~) of u E U has the mean
a(u) = IT u(t)a(t)dt = (u, a),
and the correlation function
UE U,
B(u, v) = IT IT u(s)v(t)B(s, t) ds dt = (Bu, v), u, VE U,
where the correlation operator B is given by a kernel B(s, t):
Bu(t) = IT B(s, t)u(s) ds.
(4.12)
(4.13)
The random variable ~ E 22 is Gaussian. In fact, as can be easily seen, for a continuous function u = u(t) of t E T the random variable (u, ~) = S T u(t)~(t) dt is the limit in the mean of Gaussian variables of the form
n
L U(tk)~(tk)(tk - tk - 1), k=l
where to ~ tl ~ ... ~ tn are subdivision points of an interval T and for an arbitrary function u E 22 the variable (u, ~) is the limit in the mean of Gaussian variables (un' ~) (here Un = un(t), n = 1, 2, ... , is a sequence of
12
1.5 Conditional Probability Distributions and Conditional Expectations
continuous functions convergent in the mean to the function u = u(t)). The limiting value for a sequence of Gaussian variables is, as known, also Gaussian.*
I.5 Conditional Probability Distributions and Conditional Expectations
Let ~(u) = ~(w, u), u E U, be a family of Gaussian random variables on a probability space Q and let m-(U) be the rr-algebra of sets in Q generated by all variables ~(u) = ~(w, u) on Q (parameter u runs through U). We shall assume for simplicity that M~(u) = 0, U E U. Denote by H(U) the Hilbert space of random variables '1 (measurable with respect to the rr-algebra m-( U)) with the scalar product
(5.1)
Denote by H(U) the closed linear hull of all variables ~(u), u E U. Let S and T be some subsets in U.
We shall consider an arbitrary variable '1 E H(S) and the projection fi of it onto a subspace H(T). Since H(U) is the aggregate of Gaussian variables, the difference ,1 = '1 - fi as a Gaussian variable orthogonal to all variables ~(t), t E T, is independent of all these variables. Therefore,
'1=fj+,1, (5.2)
where fj is the variable described above, measurable with respect to the rr-algebra m-(T), and ,1 is a Gaussian variable with zero mean and variance rr2 = M('1 - fi)2; this Gaussian variable is independent of ~(t), t E T. It can be easily seen t that the conditional distribution ofthe variable '1 with respect to the rr-algebra m-(T) always exists and that for almost all w it is a Gaussian distribution with the mean
M{'1Im-(T)} = fi(w) (5.3) and the constant variance
(5.4)
Let us recall that the conditional expectation M{'1Im-(T)} is geometrically the projection of the variable '1 E H(U) onto the subspace H(T), and in our case this is the variable fj, i.e., the projection onto the subspace H(T).
It is well known that Gaussian variables have finite moments of all orders. Denote by Hn( U) the closure of the subspace of all variables of the form
(5.5)
where CP(Xb ... ,xd is a polynomial of degree not higher than n in the arbitrary number of variables Xb' .. , Xb and U b ... , Uk E U.
* See, for example, [10]. p. 33.
t See, for example, [6], p. 75.
13
I Preliminaries
Theorem 2. For any variable I] E Hn(s) its conditional expectation q =
M{I]I~l(T)}belongs to the subspace Hn(T) (with the same index n):
M{I]I2l(T)} E Hn(T). (5.6)
Proof. We can consider without loss of generality that the set S and the set T are finite (say, S = {st. ... , Sk} and T = {tt. ... , td). In fact, we can go over to the general case by passage to the limit, * since
I] = lim I]m' M{I]I2l(T)} = lim M{lIml2l(T)}, m->oo m->oo
where the convergence is in the mean, and
and also
m->oo
Let I] = q{r;(Sl), ... , ((Sk)], where <p(xt. ... , x k) is a polynomial of degree not higher than n. As we have seen, the theorem is certainly true for n = 1. Assume that it is true for all indices not exceeding n - 1. Denote by ~(s) the projections of variables ((s), j = 1, ... , k, onto the subspace H(T). The differences ((Sj) - ~(Sj),j = 1, ... , k, are independent of ((t), t E T. Set
(= <p[r;(Sl) - ~(Sl)' ... , ((Sk) - ~(sdJ.
The variable ( is independent of ((t), t E T, and in the expansion
k 8 ~ I] - (= j~l 8xj <P[((Sl), ... , ((Sk)]((S) + ...
the right-hand side is a linear combination of expressions of the form
<Pm[((sd, ... , ((Sk)] 'l/In-m[~(Sl)' ... , ~(sd],
where <Pm(xt. .. . , Xk) and l/In-m(x 1 , ••• ,xd are polynomials of degree not higher than m and n - m, respectively, with m ~ n - 1. By hypothesis the conditional expectation fim = M [l]ml2l( T)] of the variable 11m = <Pm[ ((s d, ... , ((sd] belongs to the subspace Hm(T), m ~ n - 1. Obviously, the product fim . l/In-m[~(sd, ... , ~(Sk)] belongs to Hn(T) and hence also the linear com bination of these products, the conditional expectation of the difference I] - ( equal to fi - M( being such a linear combination. Therefore, fim =
M[I]I2l(T)] E W(T), as was to be proved. D
We shall define next Hermite polynomials in several variables. Let P(dx) be Gaussian measure in a k-dimensional space IRk of vectors
x = [Xl' ... , Xk] and let H be a Hilbert space of all real square integrable
* See. for example [6], pp. 29, 287.
14
1.5 Conditional Probability Distributions and Conditional Expectations
functions cP = cp(x) of x E IRk with the scalar product
< cp, 1jJ> = f~k cp(x)ljJ(x)P(dx).
As is well known* a Gaussian distribution has finite moments of all orders, and the aggregate of all polynomials q> = CP(Xb ... , Xk) of the vari ables Xb' .. ,Xk is a set everywhere dense in H. Any polynomial cp = CP(Xb ... , xk) of degree p orthogonal to all polynomials of degree lower than p will be said to be a Hermite polynomial.
Denote by H p the aggregate of all Hermite polynomials of the same degree p. Obviously, H p is a finite-dimensional subspace, and the Hilbert space H is the sum of orthogonal subspaces H P' P = 0, 1, ... :
Let us consider a Gaussian vector variable [~(ud, ... ,~(Uk)]. Denote by H p(Ub ... , Uk) the aggregate of all variables of the form
11 = cp[ ~(Ul)' ... , ~(Uk)]'
where q> = CP(Xb ... , Xk) is the Hermite polynomial of degree p with respect to the distribution P of the vector variable [~(Ul)' ... ,~(Uk)]. Obviously
00
(5.7)
Lemma 3. For any Sb ... , Sk and tb' .. , tz, andfor any p and q (with p =1= q),
the subspaces Hp(Sb"" Sk) and Hq(t 1, ••• , t l ) are orthogonal:
Hp(Sb' .. , sd ~ Hq(t b ... , t l ). (5.8)
Proof. Let p < q for definiteness. We shall consider an arbitrary variable 11 E Hp(Sb .... , Sk) and the conditional expectation q = M{I1IU(t1, ... , t l )} of this variable. By Theorem 2 the variable q belongs to the subspace HP(t b ... , t l ) = I~=o EB Hr(tb ... ,tl ). But q is the projection of the variable 11 onto the whole space H(tb ... , t l ) so that the difference 11 - ij is orthogonal to H(tb ... , t l ) and, in particular, 11 - ij -.l Hq(t b ... ,tl ). At the same time ij ~ Hq(t b .. . , t l ), since ij belongs to HP(t b ... , t l ), orthogonal to Hq(t b ... , tl )
for p < q. Consequently,
11 = [(11 - ij) + ij] ~ Hitb ... , t l ),
which was to be proved. o * This fact can be explained as follows. Obviously, the system of functions of the form eil!, x>, (t, x) = 2)kXx, is complete in a complex space H, and
I N [i(t x)]"112 0- 21"+ 1) eil!, x) - I --' ,- ,;; c -( 1)' -+ 0, where 0- 2 = f (t, X)2 P(dx),
"~[ n, n + ,
I Preliminaries
Furthermore, we shall define the subspace H p( U) as the closed linear hull of all subspaces Hp(u b ... , ud where Ul , ... , Uk E U (index p is the same for all Ub ... ,ud. Obviously, by virtue of (5.7) and (5.8) we have
n 00
Hn(u) = l: EB Hp(U), (5.9) p=o p=o
where Ho(U) contains only constant variables; Hl(U) = H(U) is the closed linear hull of variables ~(u) E U; H z(U) is the closed linear hull of variables [~(Ul)~(UZ) - B(ub uz)], where Ub Uz E U; H3(U) is the closed linear hull of variables [~(ud~(UZ)~(U3) - ~(udB(uz, U3) - ~(UZ)B(Ul' U3) - ~(u3)B(Ub uz)] where Ub Uz, U3 E U; etc.
As seen from (5.9), the conditional expectation M [l1l~(T)] of any variable 11 E H p(S) with respect to the a-algebra ~(T) belongs to the subspace H p( T) (with the same index p).
We recall the general formula for products of Gaussians:
M~(Ul) ... ~(un) = l:TI B(Ub Uj), (5.10)
where the sum is to be taken over all subdivisions of the set (u b ... , un) into pairs (Uk, Uj ), and the product is to be taken over all pairs (Ub Uj) of the corresponding subdivision.
We obtain this formula from the relation
on M~(ud ... ~(Un) = OAl ... OAn cp(O),
where cp(A) = M exp{i(A, ~)}, the characteristic function of the Gaussian vector ~ = [~(ud, ... , ~(un)]' is of the form (see (1.1))
I.6 Gaussian Stationary Processes and the Spectral Representation
A Gaussian random process ~(t) = ~(w, t) with values in a probability space Q, where the parameter t takes integer (discrete) or real values ( - 00 < t < 00), is said to be stationary if its mean is constant
a(t) = M~(t) == a
and the correlation function B(s, t) depends on the difference (s - t) only:
B(s, t) = M[~(s) - a][~(t) - a] = B(s - t) (6.1)
(in what follows we shall take a = 0). The function B(t) in (6.1) is said to be a correlationfunction ofthe stationary
process ~(t); it can be expressed as
B(t) = f eiAtF(dA), (6.2)
1.6 Gaussian Stationary Processes and the Spectral Representation
where F(dJ.) is called the spectral measure of the stationary process ~(t)
(positive bounded measure). In (6.2) the integration is over - n ,;:; J. ,;:; n in the case of discrete "time" t and over - 00 < ). < 00 in the case of continuous time t.
The stationary process ~(t) permits a spectral representation of the form
~(t) = f eiAtcI>(dJ.), (6.3)
where cI>(d),) is called the stochastic spectral measure such that
McI>(,11)cI>(,12) = F(Ll 1 n Ll 2)'
Each variable '1 from the closed linear hull H(T) of the values ~(t), t E T, permits a spectral representation of the form
1] = f q>(J.)cI>(d)'), (6.4)
where q>(J.) is the function from the space LT(F), the real linear hull of the functions eiAt of )., t E T, closed with respect to the scalar product
(6.5)
The stochastic integral given by (6.4) is defined for any function q> E LT(F) and yields '1 E H(T). The correspondence 1]+--+q>().) is a unitary isomorphism* of the Hilbert spaces H(T) and LT(F):
(1]1> '12) = (q>b q>2)F' (6.6)
In the case where the parameter t is continuous and the set T is a finite interval, we can define the space LT(F) as the closure of the subspace L 0
(by the scalar product given by (6.5)) of all functions of the form
q>().) = fTeiAtu(t)dt, (6.7)
where the u = u(t) are infinit-ely differentiable functions vanishing outside of the interval T. Since the functions q>(J.) decrease faster than IJ.I-n as J. -+ 00,
the scalar product (6.5) can be defined on the subspace L 0 with the help of a finite spectral measure as well as any (J-finite measure G(dJ.) satisfying the condition
for some integer n. Let us set
(6.8)
and define the complete Hilbert space LT(G) to be the closure of all functions of the form (6.7) by the scalar product given by (6.8).
Let LT(G) be a Hilbert space of the type indicated. (6.4) prescribes the random functional '1 = '1(q» defined on the everywhere dense subspace of
* For this see, for example, [24].
17
I Preliminaries
functions LT(G) n LT(F). We want to know under what conditions ry = ry(cp) is (to within an equivalence) a random element from the conjugate space of LT(G), i.e., a Gaussian linear continuous functional on the Hilbert space LT(G).
Suppose ry = ry(cp) is a random element from the conjugate space of LT(G), I.e.,
(6.9)
where ry = ry(A) is a Gaussian function with trajectories in the Hilbert space LT(G). The correlation operator B can then be found from the relations
<BCPb C(2)G = Mry(cpdry(CP2) = <CPb C(2)F = <Acpb Ac(2)F = (A*Acpb C(2)G,
where A is the operator on the Hilbert space LT(G) into the Hilbert space LT(F) determined by the equality
Acp(A) = cp(A), (6.10)
and where A* is its adjoint on LT(F) into LT(G). B will be a nuclear operator like any correlation operator.
On the other hand, if the operator B = A * A is nuclear by Theorem 1 the Gaussian linear functionalry = ry(cp) with correlation functional <Bcpb C(2)G is equivalent to a Gaussian element in the space adjoint to LT(G) determined by (6.9).
Let us note that we have the inclusion
LT(G) s:: LT(F)
not only for a nuclear operator but also for any bounded operator B = A * A, since
Ilcplli = <Acp, ACP)F = <Bcp, cp)G:::; IIBII·llcpll~· We note that for the finite measure G(dA), (6.9) is equivalent to a spectral
representation of the initial stationary process ~(t), t E T:
t E T. (6.11)
In fact, the functions cp(A) = eiAt are complete in the Hilbert space LT(G) and, from (6.11), ry(eiAt) = <e iAt , ry)G, t E T, extends to the whole space LT(G), the closed linear hull of functions of the type cp(A) = eiAt.
If we want to deal with the initial random process ~(t), t E T, rather than its functionalry(cp), cP E LT(G), we need to introduce the space X of all real functions x = x(t), t E T, permitting a spectral representation as
x(t) = f e-iAtljJ(A)G(dA), t E T, (6.12)
where IjJ(A) E LT(G) and the values of x(t) coincide with values of the linear continuous functional < cP, IjJ)G with cp(A) = eiAt• It is seen that (6.12) provides the one-to-one correspondence between x E X and IjJ E LT(G).
If we introduce a scalar product so that
<Xb X2) = <1jJ1> 1jJ2)G, (6.13)
18
1.6 Gaussian Stationary Processes and the Spectral Representation
where t/!l and t/!2 correspond to Xl and X2, respectively, X will become a complete Hilbert space.
If we consider the special case where G(dA) = (1/2n) dA and the Hilbert space LT(G) consists of functions of the form
(6.14)
where X = x(t) belongs to the usual L 2(T) space of real square-integrable functions with the scalar product
<Xl, x 2) = fT Xl(t)X2(t)dt,
(6.14) yields the usual Fourier transform. By the well-known Plancherel formula,
which implies that if(6.11) and (6.12) are understood in the sense ofa Fourier transform, the Hilbert space X of all square-integrable functions X = x(t), t E T, is seen formally as a special case of the general scheme of construction of a Hilbert space with scalar product given by (6.13).
We have actually proved the following:
Theorem 3. A random process ~(t), t E T, is (to within an equivalence) a random element of a Hilbert space X if and only if the product B = A * A is a nuclear operator on a Hilbert space LT(G) where the operator A is defined by (6.10).
We shall prove below that for a rather wide class of absolutely continuous measures F(dA) and G(dA) with densities f(A) = F(dA)/dA and g(A) = G(dA)/dA, the operator A * A is nuclear if
f f(A) g(A) dA < 00 (6.15)
(there exist many cases where the operator A * A is not nuclear if the condition given by (6.15) is not satisfied).
We shall examine in Chapter III an operator of the form L1 = A! A 1 - E where A! is an operator of the same type as A but which maps LT(G) onto a space LT(Gl ) constructed with respect to the measure Gl(dA) having density gl(A) = g(A) + f(A).
Since f(A) = gl(A) - g(A), we have
<L1cp, t/!)G = <A!AlCP, t/!) - <cp, t/!)G
= <cp, t/!)G, - <cp, t/!)G
19
I Preliminaries
for any 4>, tf; E L T ( G); therefore, the operator A * A coincides with LI. We prove (see Chapter III, Theorem 17) that the condition
f [I(A)]2 g(A) dA < ro
is sufficient for the operator LI = A * A to be a Hilbert-Schmidt operator. Hence, if we introduce the space LT(F 1) with 11 (A) = -/I(A)g(A), the operators of the type indicated,
B, B2 LT(G) ~ LT(F 1) and LT(F d ~ LT(F),
will be such that B!Bl and B!B2 are Hilbert-Schmidt operators under the condition given by (6.15). It can be easily seen that A*A = B!B!B2Bl is nuclear (see, for example, [7J, p. 57).
I.7 Properties of the Sample Functions*
1.7.1 Differentiability in the Mean: Some Asymptotic Relationships
Let ~(t), - Cf) < t < ro, be a Gaussian stationary process with continuous time t.
A process ~(t) is said to be differentiable (in the mean) if there exists a limit (in the mean)
This limit exists if and only if in the Hilbert space LT(F), where T =
( - ro, ro), the limit eiAh - 1 . .
lim e'At = iAe'At h~O h
exists; obviously, this is equivalent to the condition
f.':'oo A 2 F(dA) < ro (7.1)
(here F(dA) is the spectral measure of the stationary process ~(t». If
~(t) = f_oooo eiAt<P(dA)
is the spectral representation of the Gaussian stationary process ~(t), its derivative ~'(t), - ro < t < ro, being also a Gaussian stationary process will be
~'(t) = f.':'oo eiAt(iA)<P(dA).
It can easily be seen that the condition given by (7.1) is equivalent to
(7.2)
20
I. 7 Properties of the Sample Functions
as h ~ 0, where B(t) is the correlation function and .d n denotes the operator of taking the difference (.dhB(t) = B(t + h) - B(t)). In fact, by virtue of (7.2) we have for any A that
fA ),2 F(dA) ~ C fA 1 - cos Ah F(dA) ~ C .d - h.d hB(O) - A -- - A h2 "" h2
for sufficiently small h (no matter how large the prescribed A is) where C is a constant; hence (7.1) follows from (7.2).
Let us consider a nondifferentiable stationary process ~(t) and examine the conditions on the spectral measure F(dA) for which we have the relation
(7.3) where 0 < rJ. < 1.
We have
.d - h:z:B(O) = f~oo C -~~dh J (1 - codh)l -a F(dA)
~ C f~oo A2aF(dA);
f~oo A2a F(dA) < 00. (7.4)
We discuss in more detail the case where there exists a spectral density f(A) = F(dA)jdA. For sufficiently large A, IAI ?: A, let
f(A) = IAI-P
(where f3 > 1 since the spectral density f(A) must be an integrable function). For a non differentiable stationary process we must have f3 ~ 3. Let f3 < 3. Then
Substituting the variable Ah = f.1 we have*
f oo 1 - cosAh dA = hP-2a-1 foo 1 - cosu du A APh2a Ah uP
~ hP-2a-1 foo 1 - cosh du Jo uP
and, therefore,
where
* IX - f3 for the variables IX and f3 implies that lim IX/ f3 = 1.
21
I Preliminaries
It can easily be deduced from the relations thus obtained that if
f(}.) = O{I}.I- P},
(7.3) will be satisfied for 20( = f3 - 1; if
lim f(}.)I}.I P = 00, .l.~eo
we shall have for 20( = f3 - 1 that
-I' - Ll_hLlhB(O) _ 1m hZ'" - 00. h~O
(7.5)
(7.6)
Similarly, for f3 = 3 we obtain for a spectral density f(}.) of the type given by (7.5)
L1_h~~B(O) ~ c reo 1 - ~osu du + 0(1) = O{llnlhll}, JAh u
and, therefore, (7.7)
1.7.2 Continuity
Let us consider the nondifferentiable Gaussian stationary process ~(t), - 00 < t < 00, with correlation function B(t) satisfying (7.3).
Theorem 4. Under the condition given by (7.3) there exists an equivalent Gaussian process ~(t) for each trajectory of which, for sufficiently small h,
(7.8)
uniformly with respect to t in each finite interval, where C is a constant.
Proof. For sufficiently small h we have
P{ILlh~(t)1 > c'lhl"'llnllhII1/2}
= _2_ r e- x2/z dx ~ J(c'/c"Jllnlhll ' /2
~ _2_ r xe-x2/2 dx "" J2ir Jrc'lc"Jl1nlhll ' /2
= _2_ e1/2(c'lc"J21nlhl = -2-lh I P
J2ir J2n ' 1 (C')2
f3 = 2. c" '
where c" is the constant in the relation Ll-hL1hB(O) ~ c"lhl z", and c' is chosen so that f3 > 1.
22
I.7 Properties of the Sample Functions
Consider the initial process ~(t) at binary-rational points of the form t = kl2n, assuming for simplicity that 0 ,,:; t ,,:; 1. For h = 2 -n we have
Since f3 > 1 and the series I:= 1 2 -(p-l)n is convergent, it follows from the Borel~Cantelli lemma that for sufficiently small h
uniformly for k = 0, ... , 2n - 1. It can easily be seen that any interval [kI2n, kd2"'] can be taken as the
sum of intervals [rI2m, (r + 1)j2m] where rand m are integers, but not more than two intervals of the type indicated for any m. Therefore, for any h (with appropriate n within r" ,,:; h ,,:; 2 -n+ 1)
where I* denotes the summation over corresponding m. Taking this fact into account, we get for sufficiently small h
,,:; I* c'r"mlln 2m11/2 m
,,:; Qhl"llnlhlll/2.
Therefore, the trajectories of the process ~(t) considered satisfy with prob ability 1 the condition given by (7.8) on the set of all binary-rational points; in particular, for almost all WE Q the trajectories ~(w, .) = ~(w, t) are uni formly continuous functions on the set dense everywhere of binary-rational points tkn . Obviously, for an arbitrary point t the limit limtkn _ t ~(w, t kn )
exists and coincides with the initial value of ~(t) = ~(w, t) for almost all W E Q. It is seen that the trajectories of the equivalent process with values defined as
~(W, t) = lim ~(w, tkn)
23
I Preliminaries
satisfy the condition given by (7.8) for almost all w. Defining the variables ~(w, t) for the remaining w (assuming, for example, ~(w, t) == 0), we obtain an equivalent process ~(t) whose trajectories satisfy the condition given by (7.8). The theorem is proved. 0
1.7.3 Limit Theorems
Let us consider a nondifferentiable Gaussian stationary process ~(t) with correlation function B(t). Assume that on an interval (0, T) the second deriv ative B"(t) exists (with the exception ofa finite number of points) with discon tinuities of the first kind only, i.e., the finite limits B"(t - 0) = limh-+o B(t - h) and B"(t + 0) = limh-+ o B(t + h) exist within the interval (0, T) for any point t. We recall (see Section 1.1) that for the nondifferentiable process ~(t)
1. LI_ hLi hB(O) 1m h2 = 00;
h-+O (7.9)
therefore, the derivative B"(t) has discontinuities of the second kind at the point t = O. More precisely,
lim B"(h) = - 00. h-+O
Theorem 5. We have the following limiting relation:
. 1 N-l [Llh~(kh)J2 hm- I = 1, h-+O N k=O LI-hLlhB(O)
(7.10)
where h = TIN and the limit is understood in the sense of the mean convergence.
Proof. Let n be a number of points of discontinuity of B"(t). Let us take an arbitrarily small 8 > O.
Each point of discontinuity t, 0 ~ t ~ T, can be enclosed in some interval so that the total length of these points does not exceed 8. Let us denote by IE the complement of the union of these intervals (IE is the union of a finite number of intervals). It is clear that the function B"(t) is uniformly con tinuous on the set IE and
LI_hLlhB(t) = O{ B"(t)h2}
uniformly with respect to t E Ie. Taking into account (7.7) we have
LI_hLlhB(t) = o{ LI_hLlhB(O)}
(7.11)
(7.12)
uniformly with respect to t E Ie. Furthermore, since the function LI_hLlhB(t) as well as the correlation function B(t) are positive definite, it follows that for all t
(7.13)
24
We have
and, by (5.10),
M[Llh~(S)' Llh~(t)· Llh~(U)' Llh~(V)] = Ll-hLlhB(s - t)· Ll-hLlhB(u - v) + Ll-hLlhB(s - u) . Ll_hLlhB(t - v)
+ Ll-hLlhB(s - v) . Ll_hLlhB(t - u).
It is easy to calculate that
(J (h) = M - I-I Z [1 N-l Llh~(kh)Z JZ N k~O Ll-hLlhB(O)
=~ NIl [Ll_hLlhB((k-j)h)JZ. N k,j~O Ll-hLlhB(O)
(7.14)
For fixed j no more than 1 + [bh- l ] points of the form (k - j)h can belong to each interval of the length b; the total number of such points does not exceed N(l + [bh- l ]). Therefore, the number of points of the form (k - j)h in the complement of the set I, does not exceed the number (Nn + NZc/r:). It follows from (7.11)-(7.14) that for any c > 0, for sufficiently small h
where C is a constant. The theorem is proved. D
Let us note that there exists a subsequence hi> h2' ... , for which the limiting relation given by (7.10) is satisfied with probability 1. Furthermore, we can take as the subsequence hI, hz, ... any sequence for which
(7.15)
since under the condition given by (7.15) there exists a sequence Cn ~ 0 such that
{ 1 1N-I[Llh~(khW I } I P - I-I ~ Cn < 00, h~hn N k~O Ll_hLlhB(O)
and it follows from this by the Borel-Cantelli lemma that we have conver gence with probability 1 in (7.10) for h = hn • It is interesting to examine the question of the rate (defined in (7.14)) at which the function (J2(h) de creases as h ~ 0:
25
I Preliminaries
Lemma 4. Under the assumptions made before on the function B"(t) we have the following estimate:
N-l
(7,16)
Proof. It is seen that
d2(h) = 0 {h- 1[Ll_ hLl hB(0)]2 + h2 II [B"(s - t)]2dsdt}, Is-tl>2h
Further, for any fixed b > ° II [B"(s - t)]2 ds dt = 0 {I:h B"(t)2 dt},
Is-tl>2h and if the function B"(t) is monotone in a certain neighborhood (0, b) then
I:h B"(t? dt = B"(2h + 8)[B'(b) - B'(2h)],
where 2h ~ 8 ~ band
B"(2h + 8) = O{h2Ll_ hLl hB(0)},
If the function B"(t) is monotone, it retains the sign in some neighborhood (0, b); therefore, B'(t) is also monotone. Obviously, the function LlhB(t) =
S~+h B'(s) ds as well will be monotone. Hence, when iB'(2h)i--+ 00 as h --+ 0, then
iB'(2h)i = O{h-1[LlhB(h)]},
Therefore,
II [B"(s - t)]2dsdt = 0 {h- 2i Ll_ hLl hB(0)i, if B'(2h) is bounded Is-tl>2h h- 3 iLl -hLl hB(OW, if B'(2h) is unbounded.
As a result we have
d2(h) = O{max(ihi-liLl_hLlhB(OW, iLl-hLlhB(O)i)}·
The relation given by (7.16) yields the following estimate for the function (J2(h):
(7.17)
In particular, under the condition given by (7.3) the function (J2(h) de creases as h --+ ° at least as ihip:
f3 = max{l, 2(1 - ct}}.
26
1.7 Properties of the Sample Functions
Therefore the condition given by (7.15) will be satisfied, for example, for any sequence of the form hn = 2 -n, n = 1, 2, ....
For the type of stationary Gaussian processes considered, we have finally the following result, completing Theorem 5.
Theorem 6. Under the condition that
L1-hL1hB = 0{lhI 1/ 2 }
we have the limiting relation
(7.18)
1 N-1
lim h -1 - I L1h~(kh)L1h~(t + kh) = B'(t - 0) - B'(t + 0), (7.19) h~O N k=O
where t is any fixed point of the interval (0, T), N = [h -1( T - t)] - 1, and convergence is in the mean.
Proof. Elementary calculations similar to those carried out before yield that the variable
has the mean
(h) = L1hL1-hB(t) = B(t - h) - B(t) _ B(t + h) - B(t) MI1 h h h'
its variance being such that
DI1(h) ~ Cd2(h).
It is seen from the estimate given by (7.16) for the variable d2(h) that we have (7.19) under the condition given by (7.18).
Of course, the relation in (7.19) holds with probability 1 for a sequence h = hn' n = 1, 2, ... , decreasing sufficiently rapidly. D
27
11.1 Preliminaries
Il.l.l Introduction
We have already seen (in Section I.6) that the Hilbert space of random variables H(T) generated by a stationary process ~(t), t E T (with spectral measure F(d)')), is isometric to a space of functions LT(F), which is the closed linear hull of the functions eiM for). E [ - n, n] in the case of discrete time t
and for)' E [ - 00, 00] in the case of continuous time t. This fact enables us to investigate stationary processes using analytic tools. To do this, it is useful first to study in detail the analytic structure of spaces LT(F), as we shall do in this chapter, restricting ourselves to the case where T is a finite interval or a half-line.
It is clear that we may limit ourselves to the interval T = [ - T, T] or T = [0, T] and half-lines T = (- 00,0] or T = [0, (0), since an arbitrary interval or the half-line Tl can be obtained by a "shift" of T by some real t, and the corresponding space LdF) is obtained by multiplying LT(F) by eiAt •
To make it easier for the reader to grasp the results to be obtained in this case, we shall assume that ~(t) is a stationary process with discrete time with spectral density f().) = 1. It is seen that here LT(F) consists of trigo nometric polynomials P(e iA) = LIE T c(t)e iM if T is a finite interval, and in fact coincides with the Hardy space £,2 within a circle (or outside of a circle) if T is a half-line; more precisely, LT(F) consists of square-integrable func tions q>().) that can be expanded in a single Fourier series:
o q>().) = L c(t)eiAI for T = ( - 00, 0]
-00
28
cp(A) = L c(t)eiAt for T = [0, 00)).
° What is the space LT(F) if f(A) i= 1 in the case of continuous time
processes? It is useful to note initially that if the spectral measures F(dA) and G(dA)
are related by the inequality
F(dA) ;:::: G(dA)
(F(dA) majorizes G(dA)), the corresponding Hilbert spaces LT(F) and L T( G) satisfy the inclusion
This is immediate from the fact that any fundamental sequence of func tions of the form CPn(A) = Lk CkneiAtkn, n = 1, 2, ... , in the space LT(F) con vergent to a function cp(A) E LT(F), is, at the same time, fundamental in the space L T ( G):
for m, n ~ 00. In this case the limiting function I/J(A) E LT(G) coincides al most everywhere relative to G(dA) with the limiting function cp(A) E LT(F) indicated above. Therefore, the functions cp(A) and I/J(A) coincide, being the elements of the Hilbert space L T( G), i.e., cp(A) E L T( G).
This fact immediately implies that if we have a spectral density f(A) = F(dA)/dA of the type*
f(A) >< 1 (1.1)
(in the case of discrete time t), as well as a spectral density f(A) = 1, the space L[o, t](F) consists of all the polynomials
t
t=O
(with real coefficients c(t), ° ~ t ~ r). Then the spaces L(-oo, o](F) (respec tively, L[o, oo)(F)) consist of square-integrable functions that can be expanded as a Fourier series of the form
° cp(eiA) = L c(t)eiAt
-00
that coincide with boundary values (for r ~ 1) of analytic functions cp(z) in the circle Izl < 1 (outside of a circle) of the Hardy spaces £2 mentioned above:
cp(eiA) = lim cp(z), r .... 1
* We recall that for the variables a and P the relation a x P implies that 0,;;; C 1 ,;;; alP,;;; C2 < 00.
29
(1.2)
where n is an integer, is an analog of the condition given by (1.1). For n = 0 the space L[D. tiF) defined in Section 1.6 consists obviously of
square-integrable functions <p(Je) that can be expressed as Fourier integrals of the form <p(Je) = So eiAtc(t) dt. Similarly, the spaces L( - 00, D](F) (respectively, L[D, oo)(F)) consist of functions of the form <p(Je) = S~ eWc(t) dt (respectively, <p(Je) = So eWc(t)d(t)).
Making use of elementary knowledge only, we can show that under the condition given by (1.2) the space L[o, t](F) coincidls with the class of functions that can be expressed as
<p(Je) = P(iJe) + (1 + iJet Sot eiAtc(t)dt, (1.3)
where P(iJe) is a polynomial of degree less than n - 1 (with real coefficients) and c(t) is a square-integrable (real) function.
In fact, all the functions <p(Je) = (iJe)ke iAS, k = 1, ... , n - 1, being limits of the form
eiA(s + h) _ eiAS <p(Je) = lim (iJe)k-1 h '
h~D
belong to the space L[D, tiF), so that each polynomial P(iJe) = L(j-l ck(iJe)k is contained in L[D, t](F). Furthermore, the functions <p(Je) = (1 + iJet- 1
(e(l + iA)S - 1),0 :::; s :::; r, contained in L[o, t](F) can be expressed as
where
<p(Je) = (1 + iJet f~ eWcs(t) dt,
{ et for 0 :::; t :::; s, cs(t) = o for s :::; t :::; r.
It can be readily seen that the closed linear hull of functions <p(Je) = (1 + iJe)"-I(e(1 +iA)S - 1), 0:::; s :::; r, and <p(Je) = (iJe)\ 0:::; k :::; n - 1, forms the whole space L[D, t](F) (being, by definition, the closed linear hull of functions eiAS, 0:::; s :::; r) since, starting from the functions <p(Je), we can arrive at the functions cp(Je) = eiAS by means of iterated integration:
f~ (1 + iJer-Ie(l+iA)sds = (1 + iJe)n-2(e(1+iA)t - 1);
etc. It can also be seen that the linear hull of "step" functions cs(t) of the type indicated, in which the parameter s runs through the entire interval [0, r], is everywhere dense in the Hilbert space :;e2[0, r] of square-integrable functions c(t), 0 :::; t :::; r. Moreover, for functions <p'(Je) and <p"(Je) of the form
<p(Je) = (1 + iJe)n f~ eiAtc(t) dt,
30
ILl Preliminaries
where c(t) is a linear combination of all step functions cs(t), we have by the well-known Parseval equality
Ilcp'(Je) - cp"(Je)lli = f~CXJ Icp'(Je) - cp"(JeWf(Je) dJe
x f~CXJ Icp'(Je) - cp"(JeWI1 + iJel 2n dJe
= 2rc f; IC'(t) - c"(tW dt.
It is clear that the closed linear hull of all functions cp(Je) of the form indicated coincides with the class of functions cp(Je) that can be expressed as
cp(Je) = (1 + iJe)n f; eiAtc(t) dt,
where c(t) E 2'2[0, ,]. Adding to this all the functions <p(Je) = (iJe)\ ° ~ k ~ n - 1, we obtain, obviously, the space L[o. T](F).
(1.3) enables us to describe adequately the variables in the space H(T) for T = [0, ,], the closed linear hull of variables ~(t), ° ~ t ~ ,. In fact, if cJ>(dJe) is the stochastic spectral measure of a stationary process ~(t), every variable 1] E H(T) can be represented as an integral 1] = S cp(Je)cJ>(dJe) (see Section 1.6) in which cp(Je) E LT(F). It is easily seen that
1] = :t: [ak~(k)(O) + bk~(k)(,) + f; ~(k)(t)Ck(t) dt}
where ak and bk are real coefficients, ck(t) are square-integrable functions, and ~(k)(t) are derivatives in the process, k = 0, ... , n - 1.
Let us note that if the spectral density f(Je) satisfies the condition
f(Je) ~ c(1 + Je2)-n
only, the corresponding space L[o. TiF) is contained in the space LT(G) associated with the spectral density g(Je) = c(1 + Je2)-n. Hence each func tion cp(Je) E L[o. T](F) can be represented by (1.3). It is also crucial to note that (1.3) determines an entire analytic function for all complex Je. Later on (see Section IlI.4) we shall show that the space L[o. TiF) can be identified with the class of functions of the form (1.3) under the condition given by (1.2), as well as under the weaker condition
(i.e., f(Je) x (1 + Je 2)-n for sufficiently large Je only). Under the condition given by (1.2) we can easily deduce from the representation (1.3) the general formula for functions <p(Je) in the spaces L(_ CXJ, o](F) and L[o. CXJ)(F). Every function cp(Je) E L[o, CXJ)(F) is, in fact, the limit of a sequence of functions CPk(Je) E L[o, Tk](F), 'k ~ 00, that can be expressed as
k = 1,2, ... ,
II The Structures of the Spaces H(T) and LT(F)
where the sequence Ck(t), k = 1, 2, ... , is fundamental in a Hilbert space £,2(0, 00) of square - integrable functions and convergent to a function c(t), ° ~ t ~ 00, in this space. It is seen that the limiting function <p(Je) =
limk~w <Pk(J,) can be expressed as
<peA) = P(iJe) + (1 + iJet foOO eiltc(t) d(t),
where P(iJe) = lim Pk(iJe) is a polynomial of degree less than n - 1. It is also seen that this function (where c(t) E £,2(0, 00)) belongs to the space L[o, oo)(F).
Similarly, the space L( _ 00, o](F) coincides with the class of all functions that can be described by the formula
<p(Je) = P(iJe) + (1 + iJet f~oo eWc(t)d(t).
We shall investigate further the structure of the corresponding spaces LT(F) when the spectral density f(Je) does not necessarily satisfy the condition given by (1.2) but still vanishes at infinity although "not too rapidly," i.e.,
f" lnf(A) dA > - 00 for discrete time t and
f oo lnf(Je) dJe > - 00 -00 1 +Je 2
for continuous time t. In this case, retaining the earlier notation, we con sider complex spaces H(T) and LT(F).
II.l.2 Functions Analytic in a Circle
Let us denote by YfP, 1 ~ p ~ 00, the class of analytic functions <p(z) in a unit circle Izl < 1 for which
If <P E YfP, then for almost all Je E [ - n, nJ the boundary values of <p(eil) = limr~ 1 <p(reil) exist and
The space YfP is a Banach space with a norm 11<pII(P) = (S~" 1<p(eil)iP dJe)l/P. We can identify YfP with the closed subspace (in the known space* £,P( - n, n)) of all functions <p(eil ) E £,P( - n, n) for which
f:" <p(eil)einl dJe = 0, n = 1,2, . .. .
* The space £,P(a, b) consists of functions <p(A) on the interval a ~ A ~ b for which 11<pII(P) =
m I <p(AJ!P dA)'/p < 00.
32
ILl Preliminaries
We shall denote by yep the given subspace consisting of boundary values of the functions described analytic in a circle.
A function cp(z) analytic inside the circle Izl < 1 is said to be an outer function if we can represent it as
{ 1 f'" eiA + z } cp(z) = aexp -2 -.-, -lnp(A)dA , n -'" e'A - z
lal = 1,
where the real function p(A) is nonnegative and In p E 2l( - n, n). A function cp(z) analytic inside a circle is said to be an inner function
if Icp(z)1 ::;; 1 and Icp(eiA)1 = 1 for almost all A E [ - n, n J. By the Blaschke product we shall mean an analytic function B(z) of the
form
[ ~ IX - ZJPn B(z) = azP Il 11X:11 ~ ~nZ ' lal = 1,
where p, Pi> P2, ... are nonnegative integers, 0 ::;; IlXnl < 1, and the product IlllXnlpn is convergent.
Theorem ([15], pp. 98-99). An inner function cp(z) is uniquely representable as the product
cp(z) = B(z) exp - '" -.-, - Jl(dA) , { f eiA + z } -'" e'A - z
where B(z) is a Blaschke function and Jl(dA) is a singular measure.
It follows readily from this result ([15], p. 123) that any nonempty family of inner functions has a greatest common (inner) divisor.
We shall denote by D the class of functions cp(z) (introduced by V. I. Smirnov-see [23]) analytic in a circle Izl < 1 that permit the representation
{ f '" eiA + z } { 1 f'" eiA + z } cp(z) = B(z)exp - -.-, -Jl(dA) x exp -2 -.,-lnp(A)dA, (1.4) -'" e'A - z n -'" e'A - z
where B(z) is the Blaschke product, Jl(dA) is a singular measure, and p(A) ~ 0, lnp E 2l( -n, n). Therefore, class D consists of functions cp(z) that can be expressed as the product of some inner function (the inner part of cp) and some exterior function (the outer part of cp).
For each function cp(z) of class D there exist, for almost all A, boundary values cp(eiA) = limr~ 1 cp(reiA) satisfying the condition Icp(eiA) I = p(A), where p(A) is the function from the representation given in (1.4) for cp(z).
Theorem ([15], p. 80). Each function cp E yel is the product of two functions from ye2.
Theorem ([15], p. 81). For f(A) > 0 and fE gl we have f = Icpl2 where cp E ye2 if and only if lnfE 21.
33
With respect to this theorem, we note that one can take
cp(z) = exp - -.-, - lnf().) d)' . { 1 f" eiA + z } 4n -" e'" - z
Beurling's Theorem ([15], p. 145). For cp E Ye 2 the functions {zncp}, n = 0, 1, ... , generate the total class Ye 2 if and only if cp is an outer function.
Let cp(z) be a function analytic in a circle. In this case the function cp(l/z) is analytic outside of a circle. Associating in particular each function analytic in a circle with a function analytic outside a circle, we have classes D and YeP of functions analytic outside of a circle. To distinguish between these two classes, we denote, if necessary, by D+ and Yep+ the classes inside a circle and by D- and YeP- those outside a circle.
II.].3 Functions Analytic in a Half-Plane
Let us denote by yfP and D the classes of functions analytic in the upper half-plane that are images of classes YeP and D in a circle under the conformal mapping of the circle onto the upper half-plane.
We shall denote by YeP the class of functions cp(z) analytic in the upper half-plane for which
f~oo Icp(x + iy)lp dx :;::;: M < 00, y ~ 0,
where the constant M is independent of y ([15], [17]). A function cp E D is said to be outer if it can be expressed as
cp z =exp - ~-~-). ( ) { I foo 1 + )'z In p().) d } ni - 00 ). - z 1 + ).2 '
where p().) is a real function, p().) ~ 0, and Inp().)/(1 + ).2) E 21( - 00,00).
We have equality for outer functions in the inequality for Poisson's integral:
Inlcp(z)1 :;::;: l' foo lnlcp().)I d)', n - 00 (). _ X)2 + y2 z = x + iy, y > 0.
A function cp E D is said to be inner if Icp(z)1 :;::;: 1 and Icp().) I = 1 (z = ). + if.1, f.1 ~ 0).
We have for functions of class D representations of the type (1.4):
. { foo 1 + )'z } {1 foo 1 - )'z In p().) } cp(z) = e"XZB(z)exp - -occ ). _ z f.1(d)') x exp ni -(0). _ z 1 + ).2 d)' ,
where ()( is a real number, B(z) is a Blaschke function, f.1(d)') is a singular finite measure, and p().) ~ 0, Inp()')/(1 + ).2) E 21( - 00,00).
34
IL2 The Spaces L + (F) and L - (F)
The other assertions can be generalized to classes D and yep in the upper half-plane in a similar way. In particular, we have the following:
Theorem of Lax ([15J). For cp E ye2 the functions {eiAtcp(A), t:;?; O} generate all ye2 if and only if cp is an outer function.
From now on the following characteristic feature of the spaces ye2 will be frequently used.
Paley-Wiener Theorem ([15J, p. 187). Thefunction cP E ye2 (in the upper half plane) if and only if
cp(z) = fooo eiztc(t)dt, Imz:;?; 0,
where c(t) E 'p2(0, (0).
If we consider classes D and yep in the upper half-plane and classes D and yep in the lower half-plane at the same time, we shall write D + and yep + in the case of the upper half-plane and D - and yeP- in the case of the lower half-plane. Note that
'p2( - 00, (0) = ye2 + EB ye2 - .
II.2 The Spaces L +(F) and L -(F)
Let ~(t) be a stationary wide-sense process with a spectral measure F(dA). Let F = Fa + F., where Fa is the absolutely continuous component and Fs is the singular component of the measure F. Set f(A} = dFa/dk In this case we shall call f(A) a spectral density even if F #- Fa. For the sake of brevity, let us introduce some new notation:
L = L(F) = L(-oo, oolF), L - = L -(F) = L(_ 00, o](F),
L + = L +(F) = L[o, oo)(F).
Theorem 1. If ~(t) is a stationary random process with discrete time t = 0, ± 1, ... , then:
1. L + (Fs) = L -(Fs) = L(Fs); 2. L + (Fa) = L -(Fa) = L(Fa) if and only if
f" lin f(A) I dA = 00;
3. if f~" lin f(A) I dA < 00, (2.1)
35
we can write f(A) as f(A) = Ig(eiAW where 9 is an outer function of class ye2 in the circle Izl < 1.
Further,
Theorem 2. If ~(t) is a stationary random process with continuous time t, - 00 < t < (f), then:
1. L +(Fs) = L -(Fs) = L(Fs); 2. L + (Fa) = L -(Fa) = L(Fa) if and only if
f oo Ilnf(A)1 dA = (f)'
-00 1+A2 ' 3. if (2.2)
f oo Ilnf(A)1 dA -001+A2 <00,
we can write f(A) as f(A) = Ig(),W, where 9 is now an outer function of class ye2 in the upper half-plane 1m z > 0, z = A + ill.
Further,
Assertions (1)-(3) of the two theorems are, in essence, equivalent to those due to A. N. Kolmogorov and M. G. Krein in the theory of prediction of stationary random processes. * Since these assertions play a fundamental role from now on we shall sketch the proofs briefly.
Proof. Let us first prove (1)-(3) of Theorem 1. Suppose that F is a singular measure on an interval [ - n, n J, and that L - #- L. Then necessarily eiA 1= L - . Denote. by cp().) the projection of the element eiA on the subspace L -. Then eiA - cp(A) =1= 0 and eiA - cp(A) 1. L -, and
f" einA(eiA - cP)F(dA) = 0, n = 0, 1, . .. .
Since the generalized measure Fl(dA) = (eiA - cp)F(dA) is analytic, that is,
n = 0,1, ... ,
by the theorem of F. and M. Riesz ([15J, p. 73) it must be absolutely con tinuous with respect to a measure F(dA). The contradiction obtained proves (1).
As to (2), suppose J~" lIn f(A) I dA = (f). Assume contrariwise that L - #- L. Then we have again that eiA 1= L -, and if cp(A) is the projection of eiA onto
* See. for example, [24].
f:" ein2t/1(l)f(A)dA = 0, n = 0,1, .... (2.3)
Let us denote by 11<pII(p) the norm in the space 'pP( -n, n). We note that the function t/lf E 'p1( -n, n). In fact, by the Schwarz inequality,
I I t/lf I 1(1) ~ 11t/lIIFa . (1Ifll(1))1/2 < 00,
from which it follows by (2.3) that t/lf E yt'1. The logarithm of any function from yt'1 is summable; therefore,
J:" lnlt/l(A)f(A) I dA > - 00.
Further, from the elementary inequality In x < x we have
f:" lnlt/l 2(A)f(A) I dA ~ Iit/lilia < 00,
f:" In f(l)dA ~ J:" f(A)dA < 00.
Together with the ptevious inequality the above two inequalities imply lnf E .p1, contrary to the hypothesis.
The contradiction obtained proves the first part of (2); the second part of (2) will follow from (3), which we Shall prove next. By virtue of (2.1) we can write f(A) as f(l) = Ig(ei2W, where
{ 1 f ei6 + Z } g(z) = exp 4n :" ei8 _ z lnlf(8)1 d8 ,
is the exterior function from yt'2. Let <p(ei2) = <p E L +(F). This fact implies that there exists a sequence of polynomials Pn(z) such that 11<p - PnllF ---+ 0. But in this case it follows also that II<pg - P ngll(2) = 114> - P nilF ---+ ° for n ---+ 00.
Obviously, Png E yt'2+. Hence the limiting function t/I = <pg E yt'2+, that is, <p = t/J/g where t/I, 9 E H2 +. Taking the canonical representation given in (1.4) of the functions from yt'2+ and D+, we can see that <p E D+.
Conversely, let <p E D+ n L(F). Then t/I = <pg E yt'2+. The function 9 is exterior, and by Beuding's theorem (see Section 11.1) the aggregation of functions {gP} in which P runs through all the polynomials is dense in yt'2 + .
This fact implies, in particular, that we can find a sequence of polynomials Pn for which, as n ---+ 00, we have
that is, that <p E L +(F). The case L -(F) should be considered in a similar way. The proofs of (1)-(3) of Theorem 2 coincide almost completely with the
proofs of (1)-(3) of Theorem 1. In fact, we can prove with the aid of the
37
conformal mapping of a circle onto a half-plane that the generalized theorem of F. and M. Riesz is true in this case as well (see [23J, p. 209). Further, if cP E £"2 in the upper half-plane, then, necessarily, we have
Finally, in proving (3) we should refer to the Lax theorem instead of the Beurling theorem. 0
It is useful to note that, in proving (3) in Theorem 1 and Theorem 2, we obtain, in essence, the following: if the conditions given by (2.1) or (2.2) are satisfied, then
II.3 The Construction of Spaces LT(F) When TIs a Finite Interval
(2.4)
We investigated in Section 11.2 the spaces LT(F) (and therefore, the spaces H(T) generated by values of the corresponding stationary process ~(t),
t E T, with the spectral measure F(dA)), when T is an infinite interval. We shall investigate in this section the case where T = [a, b J is a finite interval.
Since the case of discrete time is trivial-LT(F) consists of trigon ometrica I polynomials of the form Ia~t~b ateiAt-we shall deal only with processes with continuous time. Furthermore, we restrict ourselves to the investigation of a process ~(t) with an absolutely continuous spectral measure F(dA) and a spectral density f(A) satisfying the condition given by (2.2). As noted above, it suffices to consider intervals of the form T = [ - a, a]. Let
L"(F) = n LT(F) T=[-a,a],a>"
and
The space L O(F) is defined by the behavior of the process ~(t) in an infinitely small neighborhood of zero; its isometric space HO = nTH T, in particular, contains all existing derivatives ~(k)(O).
Let us agree to denote by D" the aggregate of entire analytic functions cp(z), z = A + ill, of finite degree ~ (J, i.e., entire functions such that
lim R -1 max Inlcp(ReW)1 :( (J
R---+oo (J
(in particular, Do denotes the aggregate of entire functions of zero degree).
38
11.3 The Construction of Spaces LT(F) When T Is a Finite Interval
Theorem 3. If the spectral density f(A) of the stationary process ~(t) satisfies the condition given by (2.2), then* it follows that
L"(F) = D" n L(F), L O(F) = Do n L(F). (3.1)
Proof. We need to prove two inclusions:
L"(F) c D" n L(F) and L"(F) ~ Du n L(F).
We shall prove the first inclusion for all (J ~ 0, and we shall prove the second inclusion for (J = ° only. t
1. Proof of the inclusion L"(F) c D" n L(F). Given any function (fJ E L"(F), there exist functions
(fJn(A) = I ajnexp{itjnA}, j
such that 11(fJ - (fJnlIF < lin, n = 1,2, .... It is seen that all (fJn E Du+ lin' Next let us prove that at each point of the complex half-plane we have
(3.2)
for any Ii > 0, the constants C, being dependent of Ii only (and not of n). The uniformly bounded family of analytic functions (fJn is compact.
Furthermore, II(fJn - (fJIIF ~ 0, and hence (fJiz) is convergent to an entire function (fJ(z) which, under the condition given by (3.2), is a function of finite degree less than (J. It can be seen that the restriction of (fJ(z) on 1m z = ° coincides with (fJ(A).
Therefore, we need only to prove the inequality in (3.2). To this end let us estimate I (fJn(z) I along 45 degree rays through the origin, and make use of the Phragmen -Lindelof principle. t
We shall estimate first l(fJn(Z) I for Ifll ~ 1. We note that, by virtue of (2.2), f(A) = Ig(AW, where g(z) is an exterior
function of class ye2. Let us introduce the functions t/ln(z) = (fJng exp{ iZ«(J + 6)},
* We do not distinguish between the functions from D. and the restriction of them on the line !l = O. t The proof of the general case (0" > 0) is similar but rather cumbersome-the reader can find this proof in N. Levinson and H. McKean, "Weighted trigonometrical approximation on [Rl with Application to the germ field of a stationary Gaussian noise, Acta Math. 112, Nos. 1-2 (1964), 99-143. A stronger result was obtained earlier by M. G. Krein who gave the integral representation for entire functions from U(F)-see "On the basic approximation problem of the theory of extrapolation of stationary random processes," DAN SSSR 94 (1954), 13-16.
The further results of this chapter are borrowed from the paper by Levinson and McKean cited above; we have altered their proofs slightly.
I See, for example, [20].
39
0< (5 ~ lin. It is clear that l/Jn E ye2, and therefore for all J1 > 0 we have
f-"'oo Il/Jn()' + iJ1W dJ1 ~ f-"'oo ll/Jn(JeW dJe
= I/CPnl/~ ~ (I/CP/IF + ~y ~ (I/CP/IF + 1)2 = C1 . (3.3)
Further, since the function 9 E ye2 it can be represented by the Paley Wiener theorem as
z = A + iJ1, J1 > 0,
where g(u) is the Fourier transform of the function g(A). Hence for all J1 > 0
(3.4)
Denote by r R the contour consisting of a segment of a line IAI ~ R, J1 = t, and an arc resting on this segment of a circle with radius R centered at the point z = i/2, Re z ;,: t. By the Cauchy formula, if Re Zo ;,: t, and the radius R is sufficiently large, we have
Relying on the relation given in (3.4) we can easily show that the integral over the semicircle r R vanishes starting from l/J(z)!(z - zo) as R ~ 00. Con sequently, by virtue of the inequalities given in (3.3) for all J1 ;,: 1 we obtain
. 1 00 Il/Jn (u + ~)I ll/Jn(Je + IJ1)1 ~ 2n f-oo 1 . (1 )1 du
(u - A) + I - - J1 2
1 (foo 1 ( i)12 foo du )1/2 ~ 2n - 00 l/J n u + "2 du - 00 u2 + ~ = C 3 · (3.5)
Next, writing Inlg(z)1 as the Poisson integral (g is an exterior function, see Section II.l)
Inlg(z)1 = ~ foo Inlg(u)1 du, n - 00 (u - A)2 + J12
z = A + iJ1,
we find that if z = Rei6, () = n14, 3n14, then, as R ~ 00,
40
Inlg(z)1 ~Izl-~O. (3.6)
In fact, for a fixed T > 0 and z = Rei6 we have
~ fT Inlg(u)1 du = 0 (~) = 0 (~) --+ O. n -T(u-;,l+/12 /1 R
At the same time we have
/11 r Inlg(u)1 d I /1 2 + ;,.2 + 1 r Ilnj(u) I d ;Jlul>T(U-)"f+/12 u~ 2n/1 Jl ul>T1+u2 U
= ~ R2 + 1 r Ilnj(u)1 du n.}2 R2 Jlul>T 1 + u2
= R . 0(1), T --+ 00,
which proves (3.6). It follows from (3.6) that for any c > 0 and z = Rei8 (0 = n/4, 3n/4; R ;:: 1)
the inequality Ig(z)l;:: C4e-elzl is satisfied, where the constant C4 might depend on c. Noting (3.5) we can obtain the estimate
(3.7)
on the rays z = Rei9 (0 = n/4, 3n/4; R ;:: 1). Similarly, if we introduce the function t/ln -(z) = cpngexp{ -iz(a + o)},
where g(z) = g("2) E yt"2 - for z from the lower half-plane, we shall obtain the estimate
(3.8)
on the rays z = Rei8 (0 = 5n/4, 7n/4; R ;:: 1). We shall take next the estimate of ICPn(z)1 on segments of the rays lying
within a circle Izl ~ 1. We assume, as usual, that In + a = In a if a > 1, and In + a = 0 if a ~ 1. Estimating the subharmonic function Inlcpn(z)1 with the aid of the Poisson integral, we have on the lines 11m zl = 1 that
lnlcpn(z)I ~ ~ foo Inlcpn(u)1 du n - 00 (u - ),,)2 + 1
1 u2 + 1 foo In + ICPn(u)1 d ~ ~ sup u
n u (u-),,)2+1 -00 u2 +1
~ ~ ),,2 + 2 foo In+lcpn(u)1 du n 2 - 00 u2 + 1
In + ICPn(u)llg(u)1
= ~ ),,2 + 2 foo Ig(u)1 du n 2 - 00 1 + u2
41
Therefore, the function e- C7z2 CPn(z), analytic in a strip 11m zl:( 1, is bounded in this strip and satisfies on its boundaries the inequality le- C7Z2 cpn(z)1 :( Cs, where Cs is independent of n. According to the Phragmen-Lindelof prin ciple the last inequality holds true at each point of the strip indicated. In particular, all the functions CPn(z) are uniformly bounded in the circle Izl :( 1 From this, and from (3.7) and (3.8), it follows that the inequality given in (3.2) holds true, which proves, as mentioned above, the first part of the theorem.
2. Proof of the inclusion L o(F) => Do n L(F). Let cp(),) E Do n L(F). By the familiar Hadamard theorem* the function cp(A), the entire function of zero degree, can be written as the product
(3.9)
where the Zn =I- 0 are zeros of cp(z). The function cp(z) can be rewritten as the sum CPl (z) + IPz(z), where cP l(Z) = i(cp(z) + cp( - z)) and cpz(z) = i(cp(z) - cp( - z)) are even and odd, respectively. In this case, we have as before that CPt. cpz E Do n L(F). Therefore, we need only to prove the theorem for even functions and odd functions. In both cases the proofs are identical, and for the sake of definiteness we shall consider the even functions cp(A).
We need to prove that for any 8 > 0 there is a function CPo E U(F) for which Ilcp - lPellF :( 8.
Let us note first that each square-summable entire function cp(A) of finite degree :( 8 belongs to U(F). In fact, each function of this kind belongs to L(F). By the Paley-Wiener theorem t the function cP has, relative to entire functions from !£'Z, the Fourier transform rp equal to zero outside of [ - 8, 8 J. We infer that
and is finite, IP E U(F). Therefore, it suffices to construct a square-summable integral function of
degree :( 8 adequately approximating cpo We note that the Hadamard fac torization given by (3.9) for the even function cP will be of the form
m~O.
We shall define the function lPo(A) by the equality
CPo(A) = AZm n (1 -A:) I1 (1 _ AZ~Z), IZnl<d Zn n>db n
where () = lOin; the number d = d(8) will be defined later on.
t See, for example, [15J, p. 82.
42
We shall show next that CfJe is a square-summable integral function of degree::::; G (and, therefore, CfJe E L 2(F)). The Euler formula
00 ( 22) sin n2 = n2 If 1 - n2
enables us to write CfJe as
(3.10)
Therefore CfJe(2) is an entire function of degree nb = G. We estimate next the ratio of polynomials in the right-hand side of (3.10) for large 2. Let us introduce the monotone nondecreasing function N 'P(R) equal to the number of roots of the function CfJ(z) in a circle Izl ::::; R. The function N 'P(R) is closely related to the growth order of the function CfJ(z). In particular, N CfJ(R) = oCR), R ~ 00,* for function

gaussian random processes

Documents