E L S E V I E R Statistics & Probability Letters 25 (1995) 329-339

STATISTI~.,S &

On weak lumpability of denumerable Markov chains

J a m e s L e d o u x *

INSA-IRISA, Campus de Beaulieu 35043 Rennes Cedex, France

Received October 1993; revised October 1994

Abstract

We consider weak lumpability of denumerable Markov chains evolving in discrete or continuous time. Specifically, we study the properties of the set of all initial distributions of the starting chain leading to an aggregated homogeneous Markov chain with respect to a partition of the state space.

Keywords: Weak lumpability; Positive recurrence; R-positivity; Quasi-stationary distribution; Uniform semi-group

1. Introduction

Let us consider a homogeneous Markov chain X, in discrete or continuous time, on a countably infinite state space denoted by E, which without loss of generality we assume to be a subset of the natural numbers I%1 (i.e. E C I%1.) Let ~ = {B(0),B(1) . . . . } be a fixed partition of E. We associate with the given chain X the aggregated chain Y, over the state space F = {0, 1 . . . . }, defined by:

Yt = l ,~==> Xt E B(I), for any t.

We are interested in the set of all initial distributions of X which give an aggregated homogeneous Markov chain Y. If this set is not empty, we say that the family of Markov chains sharing the same transition semi- group is weakly lumpable. Most of the literature on lumpability has been concerned with the strong lumpability situation, that is, when any initial distribution leads to an aggregated homogeneous Markov chain. To the best of my knowledge, the weak lumpability problem with countably infinite state space has been addressed only recently in Ball and Yeo (1993) for (irreducible positive-recurrent) continuous time Markov chains. The purpose of this note is to propose some results in discrete or continuous time, prolonging the studies reported in Rubino and Sericola (1989, 1991, 1993) and Ledoux et al. (1994) for a finite state space. Section 2 deals with discrete time Markov chains and mainly concerns weak lumpability for irreducible positive-recurrent or R-positive chains. In particular, we discuss the ergodic interpretation of the quasi-stationary distribution. The third section shows that lumpability for any denumerable continuous time Markov chains with a uniform

Regional Council of Britanny under Grant 290C2010031305061.

* E-mail: ledoux@{univ-rennes 1 .fr} {irisa.fr}.

0167-7152/95/$9.50 (~) 1995 Elsevier Science B.V. All rights reserved SSDI 0167-71 5 2 ( 9 4 ) 0 0 2 3 8 - X

330 J. Ledoux/Statistics & Probability Letters 25 (1995) 329-339

transition semi-group can always be replaced in the discrete time context. The sequel of this result are also discussed for irreducible positive-recurrent or 2-positive continuous time Markov chains.

By convention, vectors are row vectors. Column vectors are indicated by means of the transpose operator (.)*. The vector with all its components equal to 1 (resp. 0) is denoted merely by 1 (resp. 0). The set of all probability distributions on E will be denoted by ~'. For any subset B of E and ~ E ~¢, the restriction of • to B, i.e. the vector (~(i),i E B), is denoted by ~B; if ~BI* ~ 0, ~8 is the vector defined by ~B(i) = ~(i)/~'~jeB ~(J) i f i c B and by 0 i f / ~ B .

2. Weak lumpability in discrete time

Let X = (Xn)~>0 be a homogeneous Markov chain over state space E, given by its transition probability matrix P = (P(i,j))i.j~e and its initial distribution ~; when necessary we denote it by (~, P). Let P(i, B) denote the transition probability of moving in one step from state i to the subset B of E, that is P(i,B) = ~ j e s P ( i , j ) . We denote the aggregated chain constructed from (~,P) with respect to the partition ~ by agg(a,P, ~) .

Definition 2.1. A sequence (B0,B1 . . . . . Bj) of subsets of E is called possible for the initial distribution ~ iff P~(Xo E Bo,Xi ¢ BI . . . . . Xj E Bj) > 0. Given any distribution ~ E ~¢ and a possible sequence (Bo,B1 . . . . . Bj) for ~, we can define the vector f (~ ,Bo,B1, . . . ,B / ) ¢ sg recursively by:

f ( ~ , B 0 ) = ~ B°,

f(~,Bo, B1, . . . ,Bk) = (f(~,Bo, Bt . . . . . Bk-1)P) ~, k ~ 1.

For any B ¢ M, ~¢(~,B) denotes the subset of all distributions of the form f(~,B0, . . . ,Bk,B).

By definition, the aggregated chain Y =agg(a,P, ~ ) is a homogeneous Markov chain if and only if VI, m E F, Vn~>0 and V(Bo, BI . . . . , B~_t,B(I)) possible for ~,

P~(Xn+I • B(m)IX, e B(I), Xn-1 E B , - i . . . . . Xo e Bo) = P=(X,+I E B(m)[X, E B(l))

and the probability in the right-hand side does not depend on n; in that case, it describes the probability of going from state l to state m in one step for the aggregated chain ao#(~,P,~). The approach developed in Kemeny and Snell (1976) and in Rubino and Sericola (1989) consists in rewriting the above conditional expression as

P~(X1 E B(m)) with fl = f (~,Bo . . . . . Bn_I,B(I)),

that is, in including the past into the initial distribution. In the same way as in Kemeny and Snell (1976), a necessary and sufficient condition for Y to be a homogeneous Markov chain can be exhibited without any particular assumption on X.

Theorem 2.2. The chain Y = aoo(~ ,P ,~) is a homogeneous Markov chain iff Vl, m E F, the probability P#(X1 E B(m)) is the same for every fl c .rg(~,B(l)). This common value is the transition probability for the chain Y to move from state l to state m.

The aim of this section is to study the properties of the set of distributions

s~.a = {or C d /agg(a ,P, ~ ) is a homogeneous Markov chain}.

J. Ledouxl Statistics & Probability Letters 25 (1995) 329 339 331

2.1. Weak lumpability for irreducible positive-recurrent Markov chains

Throughout this subsection, we assume that the considered Markov chain is irreducible positive-recurrent. Therefore, there exists a unique probability vector, denoted by 7z, which satisfies ~zP -- zt. Let g be a real function on E and m a probability measure on E; g is m-integrable if

m(Igl) £ ~m(i)lg(i)l :E,,,[Igl] < ~ . i C E

For such a Markov chain, we have the following standard corollary of the ergodic theorem.

Result 2.3. For any bounded real function g on E, we have for all ~ E ~¢ff

lim _l ~ ~[g(Xk)] = 7t(g). n - - - * o c n k=l

We only need the following lemma to derive Theorem 2.5 from Theorem 2.2 with similar arguments as for Theorem 3.5 in Rubino and Sericola (1989).

n Lemma 2.4. Let [3~ be the vector (1/n))-~k= 1 ctP k. For any bounded real function g on E, we have for any I E F ,

lim f ( f ln ,B(l))(g) = f (~ ,B( l ) ) (g) . n --..* oo

In particular, we have for any m E F

lim ~f~/~,,.eCt))(X1 E B(m)) = P f(~,B(t))(X1 E B(m)).

Proof. To obtain the first limit, it suffices to let in Result 2.3: or(i) = g(i) (resp. 11(i) = 1) if i E B(l) and 0 otherwise. Since the numerator and the denominator of f ( f l , ,B( l ) ) (g ) = fln(gt)/fl,(lt) tend respectively to ~z(gt) and to 7z(lt), we obtain f ( l t , B(l))(g) as limit of the quotient.

In particular, if we choose as bounded function g on E: g(i) = P(i ,B(m)) for i E B(l) and 0 otherwise, then the probability

Pf(~,,.B(I))(X1 E B(m)) = f([3. ,B(l))(9)

tends to f ( n ,B ( l ) ) (g ) = Pf(~,ett))(X1 E B(m)). []

Finally, we have

Theorem 2.5. I f ~¢~ ~ 0, then rc E sd.ct and the transition probability matrix of the homooeneous Markov chain agg(~,P,~), denoted by P, is the same for all ~ E ~(,~t. The entries of matrix P are given by

P(l,m) = ~ zrs(t)(i)P(i,B(m)), l,m E F. i E B ( I )

The unicity of matrix P for all ct E ~.1¢ gives the convex property to the set ~ . In particular, if we construct the convex envelope of the family of vectors {Tt B(I), l E F}, d ~ = ~ t e F '~lrtB(Z) (with 2z >10 and ~t~F At = 1) and ~¢~t ~ 0, then we have z~¢~ C ~.~t. With the previous result, Theorem 3.7 from Rubino and Sericola (1989) can be extended to our denumerable context. Consequently, the set ~,~t is the (a priori) infinite intersection of a decreasing sequence of convex sets, denoted by ~¢J ( j ~> 1 ), which are the solutions

332 J. Ledoux/Statistics & Probability Letters 25 (1995) 329-339

to the linear systems defined as in Rubino and Sericola (1989, 1991). It can be noted, as in Rubino and Sericola (1991), that the property of P-stability of d j (i.e..~¢JP C ~¢J) allows us to identify . ~ a as the set ~ J . The example of Subsection 2.4 shows that the infinite intersection of sCJ's can be finite and explicitly computed.

2.2. Weak lumpability o f R-positive Markov chains

We are now concerned with denumerable Markov chains with absorbing states which are assumed to be collapsed in only one class (state labeled by 0 for the aggregated process Y) of the partition M. The other classes constitute a partition of the set of transient states, denoted by T, of X. It is easy to convince ourself that weak lumpability for such a Markov chain reduces to weak lumpability of the Markov chain with only one absorbing state and absorption probabilities equal to P(i, B(0)) for i E E. Consequently, we consider only one absorbing state denoted by a (and B(0) = {a}). Let us denote by ~ a r (resp. d r ) the subset of z¢~ (resp. o~') composed by the distributions • with support T, i.e. ~-]i~r ~(i) = 1. We have ~ t t = ( 1 - 2 r ) 1 {a) + 2 r ~ r where 1 ~>2r>~0. Therefore, we restrict the analysis to the set s~.a r .

In discrete time, the transition probability matrix P can be decomposed as follows: (1 ;) e----

I - O)l*

where matrix Q is assumed to be irreducible. In this subsection, we recall (e.g. see Seneta (1981, Chap. VI)) the definitions and the main properties of the R-classification of a non-negative irreducible matrix. It can be shown that all the power series Qij(z) = ~ = o Qk(i,J) zk (i,j E T) have a common convergence radius, denoted by R, which is usually called the convergence parameter of matrix Q. If T is a finite set, then R is the inverse of the spectral radius of Q. Matrix Q is said to be R-recurrent if and only if all the series ~ k Qk(i,J) Rk are divergent. Furthermore, if no sequence (Qk(i,j)Rk)k>. o tends to 0, then the matrix is said to be R-positive. For an R-recurrent matrix Q, there exists a unique (up to a constant) R-invariant measure, (resp. R-invariant vector) denoted by v (resp. w), that is

R v Q = v (resp. R Q w * = w * ) .

We can now define the stochastic matrix P whose entries are given by

w( j ) P(i , j ) z~ R ~ Q(i,j), i , j c T.

Denoting the diagonal matrix with generic diagonal entry w(i) by W, the previous relation becomes

f i = R W -~ QW. (1)

It is easy to show (as in Seneta (1981, Theorem 6.4)) that matrix P is positive-recurrent if and only if matrix Q is R-positive. The stationary probability vector of P is ~ = (v( i)w(i)) icr which gives a second characterization of the positive recurrence of P: Y~i viwi < ~ . It is important to note that the R-recurrence property does not allow in any way to infer the convergence of the series ~ k vk or Y]~k wk. It was shown in Ledoux et ai. (1994) that using quasi-stationary distribution can be fruitful for weak lumpability of a finite absorbing Markov chain. We propose in this subsection to extend some of those ideas to a R-positive Markov chain.

A quasi-stationary distribution is a probability measure on T which makes stationary the following condi- tional probabilities: for i E T, P~(X, = i IX, 6 T), that is the vector (P~(X, = i IX, E T))i~r is independent of n. The existence of such a measure under milder conditions than R-recurrence is discussed in many recent

J. Ledoux / Statistics & Probability Letters 25 (1995) 329 339 333

papers. But it can be seen that R-recurrence is nearly a "minimal" assumption (up to Harrys Veech conditions, e.g. see Pruitt (1964)). The R-positivity property of matrix Q is also the nearly "minimal" condition to have an ergodic interpretation of such a quasi-stationary distribution with any probability vector as initial distribution of the Markov chain X. Moreover, the results must include the finite state space ones reported in Ledoux et al. (1994). The following theorem gives an ergodic interpretation to the R-invariant measure v when it defines a probability distribution. Note that we do not make any distinction between periodic and aperiodic cases. Throughout the remainder of this subsection, we will assume that any initial distribution ~ E ~¢r satisfies a constraint o f the type:

~r <C~v, (2)

where C~ is a positive scalar. Since we have 0 < arWl* <~C~ vWl* = C~ 7tl* = C~, the vector (~TW)/ (a r Wl*) defines a probability distribution.

Lemma 2.6. Let g be a real function on T assumed to be n-integrable. For any initial d&tribution ~ E d r with ~r <~ C~v, we have

1 n lim - ~ ( ~ r W P k ) ( g ) = ~rWl* rt(g).

n - - - - ~ n k = l

Proof. From Result 2.3, we have that for any i E T,

lim ~rWl* n k~= 1 \ ~ - ~ 7 j (i)g(i) = arWl* x(i)g(i).

Moreover, condition (2) required on a gives the following inequality for any i E T:

(etrW~ k@lfik) (i)g(i) <~C~ ( ~ k~=l~Zfik) (i)[g(i), = C~ ~(i),g(i),.

Since ~ i E r 7r(i) ]g(i)] = n([g]) < CX~, the dominated convergence theorem allows us to write

lim ~ - ~ ( ~ r W ! k ~ P k ) ( i ) e ( i ) = ~ c t r W l * ~ ( i ) 9 ( i ) = ~ r W l * ~(9 ). D n~cx~ iET iET

Theorem 2.7. Let Q be an R-positive matrix such that its R-invariant measure v satisfies vl* < oo. Assume that c~ is a probability distribution which verifies relation (2). I f we define the vector

~=lRkc t rQ k pn.2t n ~k=l Rk~rQ k l*

then we have for any v-integrable function g on T

v(g) lira p. ,~(g) - .

n~vc vl*

This result can also be derived from Seneta and Vere-Jones (1966) but we use here standard arguments on regular Markov chains which give insight into the considered assumptions.

Proof. From definition (1) of matrix fi, we have

( O ~ T m E n k = l p k ) m -1

p,,,a =

334 J. Ledoux/Statisties & Probability Letters 25 (1995) 329-339

Let h = W-lg * and s = W - t 1"/>0, these two functions are n-integrable since we have rc(Ihl)= v(Igl ) and n(s) = vl*. The Lemma 2.6 allows us to write for all initial distribution such that aT <~C~v:

iim p,.~(g) = lim (ccrW~-]~=l pk)(h) -- CCTWI* rt(h) _ v(g) [] n ~ o¢3 n ---~ o¢) m n . (aT TWl vl*

When the state space E is finite, it is clear that relation (2) is always satisfied and the convergence in Theorem 2.7 holds for any initial distribution. Under the assumptions of Theorem 2.7, we can derive an analogous result to Theorem 2.5. Firstly, from Theorem 2.7, the same proof as for Lemma 2.4 allows us to establish the lemma:

L e m m a 2.8. For any distribution ~ E sff r satisfying constraint (2), let ft, be the vector

~']~=IRkCcrQ k [3n =

~ = I R k ~ T Q k 1 *"

Then, for any bounded real function g on T, for all l # 0 and m E F, we have the same conclusions as in Lemma 2.4.

The set {~ c ~¢r/~r <~C=v and agg(~,P,M) is a homogeneous Markov chain} is denoted by ~ ¢ ( v ) . We are in a position to show the following result:

Theorem 2.9. Let v be the quasi-stationary distribution associated with the R-positive Markov chain X. I f s e t ( v ) # 0 then (0, v)E sff~(v). Moreover, i f P denotes the transition probability matrix of the homogeneous Markov chain agg(~,P,~) then this matrix is the same for all ~ c sfff (v).

Proof. Let ~ E d r satisfying (2) such that agg(~,P, ~ ) is a homogeneous Markov chain with transition prob- ability from state l to m denoted by P(l,m). Let ~k be the vector

~TQ ~1 J "

For any k such that P~(Xk E B(I)) > 0 (l # 0), we have:

P(l, rn) = P~(Xk+I E B(m)IXk C B(I))

= P(~rQk)8,~,(XI c B(m))

= P~k,,,,(X~ G B(m)). (3)

Choose no large enough such that Vn>>-no, ~=l(Rk~rQk)e(t) l * > 0 (T is irreducible). In denoting by 7k (k = 1 . . . . . n ) the scalar

(Rk ~rQ k )8(I) l *

~ = 1 (Rk ~rQ k )B(t) 1 * '

J. Ledoux/ Statistics & Probability Letters 25 (1995) 329 339 335

the relation (3) can be rewritten as

P ( l , m ) = ~ 7k P~kB'"(X1 E B(m)) [ <<.k ~ t t ,

P;t(~) EB{/)}>'0

= Pr(X1 E B(m))

where

r = ~ ~k(~k)B(~) = (/~.)~(l~.

Pxl,~k EB[I)}>0

Therefore, we obtain

P(l ,m) = Pf(Cc,,,B(l))(Xt E B(m)).

As n goes to infinity, we derive from Lemma 2.8 that the transition probabilities of the aggregated chain are (independent of a and) given by

P ( l , m ) = ~ vB(t)(i)P(i,B(m)), l ¢ 0 , m E F . [] (4) iEB(I)

The convexity of the set ~ r ( v ) follows from the unicity of the transition probability matrix for all the Markov aggregated chain.

Corollary 2.10. I f si f t(v) ¢ 0 then s i r ( v ) is a convex set and it necessarily includes the convex subset S~¢v -~ ~IEF f~t UB(I) with 2t >~0 and ~IEF ,~1 = 1.

By definition, the set d r ( v ) is a subset of ~ r . If ~gr(v) :fi 0 we trivially have ~ '# ~= O. The converse is true at least in the following specific cases which ensure that (0, v) E ~ r .

Corollary 2.11. I f the set ~ r includes a distribution with finite support or i f there is a class B(l) (l ¢ O) within the partition ~, which collapses a finite number of states, then (0, v) E d r and we have s~¢ 0 +~. J~,(v) ¢ O.

2.3. Quasi-stationary distribution as a distribution of reset after absorption

In this subsection, we will show that the set d c~(v) is non-empty if and only if the set ~ a associated with a positive-recurrent chain is not empty too. Under the condition vl* = 1, where v is the R-invariant measure of the R-positive matrix Q, we can define the following transition probability matrix denoted by p(v):

( 0 .) p(") = (I - Q)I* Q

Lemma 2.12. The Markov chain with transition probability matrix p(v) is irreducible and positive-recurrent. Its invariant probability measure is given by

~Z(c) ---- )~R 1 {a} -[-(1 -- ,~R)(0, V) with 2R = ( R - 1 ) / (2 R- 1). (5)

Proof. The convergence parameter of the R-positive matrix Q is such that R > 1 (see Seneta (1981)). Let us consider the "taboo" probability denoted by c(k) and defined by the probability of going from state 0 to .,'00

336 J. LedouxIStatistics & Probability Letters 25 (1995) 329 339

state 0 in k steps without revisiting state 0 in the meantime. The irreducible matrix p(v) will be recurrent if and only if ~--]~,~>I JO0¢'(1') = 1. Since v is R-invariant, we have ~k~>. f~o~ ) = ~-]~k~>, vQ*-l( l - Q ) I * = ~k~>l(l/R)k-l(1 - ( I / R ) ) = 1. Finally, the positive recurrence follows from checking that the invariant

probability measure of matrix pO,) is given by formula (5). []

We can now show the main result of this subsection.

Theorem 2.13. I f s~cr(P (~)) is the set of all #zitial distributions c~ such that agg(~,P (0, 3 ) is a homogeneous Markov chain, we have ~ff.~t(v) ¢ 0 ¢=~ ~a(P(~)) y~ 0; in that case, we have ~r(v)C_ d cc(P (~)) _C s¢~¢.

The transition probability matrix p(v'--~ of agg(u,P(V),~) is given for every m E F by P(~-"~(l,m) = P(l ,m) with l ¢ 0 and by P(r)(O,m) = VB(m)I* (matrix P is given by relation (4)).

Proof. The above one to one correspondence between the respective entries of matrices P and P(~"~ is deduced from relation (5) in the previous lemma and from the definition of matrix P~).

The inclusion of d~t (P (v)) in J c r follows in the same manner as in the finite case (see Ledoux et al. (1994)) and is not reproduced here. We have only to prove that if ~ t ( P (v)) ~ 0 then d.~(v) ~ 0. Indeed if ~ff.a(P (O) ~ 0 then it (~) E J a ( P (~)) C_ d . a from Theorem 2.5. It easily follows that (7~(v)) T = (O,v) E ~%¢T and therefore, that ~ r ( v ) ¢ 0.

The proposition (~ C Jfff(v) ~ ~ c ~ff.~,(P(~))) results directly from the proof of the inclusion ~ C_ ~ffr (P(V)) in the finite case which can be found in Ledoux et al. (1994). []

We have already noted that z~,a = 2 1 {a} +(1-2)~ff.~ and that, in the finite case, ~ ( v ) = ~ . Therefore, the two sets J . a and ~¢¢(P(~')) are identical. We are not able to establish the same equality in the denumerable case. Another important fact is that the two sets 2 l{d + (1 - 2 ) j r ( v ) and ~ I t ( P (v)) are distinct in general (this will be illustrated in the example.) The equality will hold only in the case where any distribution in s !a(P ~)) can be majorized by a multiple of the stationary distribution 7r (~) of P(~).

2.4. Example

Let us consider the following partition ~ = {B(0) = {0},B(1) = {i>~ 1}} of the state space E = [~. The transition probability matrix P is given by:

P(O,O) = 1

P(1,o) o

P(n,O) 7/8

for any n >~ 2

P(0,1) = 0

P(1,n)=(1/6)(5/6)"- '

P(n, 1)= 1/8

for n ~ 2

P(0,n) = 0 for n~>2

for n~>l

P(k,n) = 0

for k >~2, n~>2.

The submatrix Q of transition probabilities between transient states (here, T = B(1 ) = {i >~ 1 }) is clearly irreducible. We deduce that ~-']~k~>l f(l] )z* = (1/6)z + (5/48)z z (with the same notation as in the proof of Lemma 2.12). It follows from Seneta (1981, Definition 6.2) that state 1 is R-positive with R = 12/5 and therefore that all the transient states are R-positive too. The 12/5-invariant probability measure v is given by

v ( 1 ) = ~ , v(n)= ~ Vn~>2.

J. Ledoux/Statisties & Probability Letters 25 (1995) 329-339 337

We can directly check that (0, v) E ~ r . Indeed, we have only one transient class B(1). The aggregated chain is a homogeneous Markov chain if and only if the distribution of the sojourn times in this class B(I ) is geometric with parameter P(1, 1) = 5/12; this is immediate because vector v is precisely a 12/5-invariant measure associated with matrix Q.

Let us consider matrix P(~) which has the following vector as invariant probability measure: rt (~') = (7/19)1 {~/ + (12/19)(0, v) (relation (5)). We can verify (construct the convex set s/I as defined in Rubino and Sericola (1991) and check its P-stability) that

, ~ a ( P (r)) = {uE,~¢/3~(1) = 1 - ~(0)}.

We can note that N r ( v ) C Ju(P(~)) . Indeed, choose a E .~r such that

~(0) = 0, ~(1) = 1/3, co(n) = \ 1 2 , / Vn~>2.

Since 3c¢(1 ) = 1 - a ( 0 ) , we have a E J a ( P (')) but the ratio ~(n)/v(n)cx (11/10) "-I is unbounded as n goes to infinity. Therefore, vector :¢ cannot satisfy relation (2), so ~ ~ ,~¢T(v).

3. Weak lumpability in continuous time

The weak lumpability property has been recently addressed in Ball and Yeo (1993) for denumerable ir- reducible positive-recurrent Markov chains evolving in continuous time. Their main result (Theorem 2.3) is the counterpart of Theorem 2.5 in continuous time. Here, we propose to briefly discuss weak lumpability for denumerable Markov chains with the only assumption of having a uniform transition semi-group denoted by (Pt)~o (e.g. see Freedman, 1983) The generator of such a Markov chain is denoted by A and it is uniformly bounded. In particular, any finite Markov chain has a uniform transition semi-group. The Markov chain (Xt)t/>0 is stochastically equivalent to the one with transition semi-group

-at (at)" U" = e - - where a >~ sup{i 'JA(i, i) j } and U I +A/a. n=0~ n[

The discrete-time Markov chain (U,),~>0 with transition probability matrix U is usually called the "uni- formized" chain associated with (Xt)t>~f0. In Ledoux et al. (1994), the result showing how to reduce the weak lumpability property from continuous time to discrete time is proved in the finite state space context. The proof is direct, avoiding preliminary works as in Rubino and Sericola (1993) (irreducible case.) Since the statement is only based on the definition of the Markov property and in the previous stochastic equivalence, this scheme still holds in the denumerable state space case. Therefore, we just express the result omitting the proof.

Theorem 3.1. Let X be a Markov chain with a uniform transition semi-group and generator A. The chain agg(~,A,~) is a homogeneous Markov chain iff agg(~, U,M) is also a homogeneous Markov chain. So we have

~da ~ {~ E ~¢/agg(~,A,~) is a homogeneous Markov chain} = •a (U) .

l f ~ E ~a then the Markov chain agg(~,A,~) has a generator, denoted by A, which is given by A = a ( U - l ) , where U is the transition probability matrix o f agg(~, U, ~).

This result allows us to derive the unicity of the generator A for all aggregated Markov chains under the assumptions of Theorems 2.5 or 2.9 for the (discrete time) "uniformized" chain (U,),~0. Specifically,

338 J. Ledoux I Statistics & Probability Letters 25 (1995) 329- 339

if (Un)n~>0 is R-positive then the continuous time Markov chain (Xt)t~>0 is 2-positive (in the terminology proposed by Kingman, 1963) with 2 = a(1 - 1/R) (see Buiculescu, 1972). Finally, we obtain

Corollary 3.2. Let X be a Markov chain with a uniform transition semi-group and generator A. 1. Assume that X is irreducible positive-recurrent with invariant probability measure ~. I f agg(~,A, ~ ) is

a homogeneous Markov chain then it admits the generator A given by

A(l,m) = ~ ~zs(t)(i)A(i,B(m)), Vl, m E F. iEB(I)

2. Let X be a Markov chain with an irreducible transient class T and all its absorbing states are collapsed in the class B(O) of the partition ~. The chain X is assumed to be 2-positive with a 2-invariant probability measure v. For any initial distribution ~ such that ~T <<, C~v, where C~ is a positive real, i f agg(e ,A ,~) is a

A homogeneous Markov chain, then its generator A is given by

A(l ,m) = ~ vBIt)(i)A(i,B(m)), Vl E F \ {0}, Vm E F. i c B ( l )

Finally we have

~u(v) ~ { ~ E d / ~ r <~C,v and agg(e ,A,~) is a homogeneous Markov chain}

= { ~ E t i l e r <~ C~v and a99(~, U, ~ ) is a homogeneous Markov chain}•

We note that Corollary 2.11 can be expressed in the continuous time context. Another remark is that the first part of the previous corollary, is stated under milder conditions in Ball and Yeo (1993, Theo- rem 2.5). We end the discussion by pointing out that the equivalence between discrete time and continu- ous time using the uniformization technique, is also reported in Sumita and Rieders (1989) for finite er- godic Markov chains• But it is based on an erroneous characterization given in Sumita and Rieders (1989, p. 66) of the weak lumpability property for discrete time Markov chains• In fact, they characterize the markovian property of the aggregated chain in terms of the Chapman-Kolmogorov equation associated with. This equivalence is false in general and has been the purpose of famous counterexamples. For instance, we can take back the irreducible transition probability matrix P considered in Rosenblatt (1971, Chap. 3, Section 1 ):

p =

1/4 1/2 0 1/4~

J 1/4 1/4 1/4 1/4

1/4 0 1/2 1/4

1/4 1/4 1/4 1/4

with stationary distribution n = (1/4, 1/4, 1/4, 1/4). The partition is composed of B(0) = {1,2} and B(1) = {3,4}. We can verify after some algebra that the condition from Sumita and Rieders (1989) is met for the initial distribution n. But the chain agg(n ,P,~) is not markovian since

E B(0), X, E B(0), Xo E B(0)) = ¢ = + g(2))(P(0, 0)) 2

with P(0, 0) = 5/8. Since matrix P is irreducible, no initial distribution can lead to an aggregated markovian chain by virtue of Theorem 2.5.

J. LedouxlStatisties & Probability Letters 25 (1995) 329 339 339

References

Ball, F. and G.F. Yeo (1993), Lumpability and marginalisability for continuous-time Markov chains, J. Appl. Probab. 19, 518-528. Buiculescu, M. (1972), Quasi-stationary distributions on continuous Markov chains, Rev. Roumaine de Math. Pures Appl. 17, 1013-1023. Freedman, D. (1983), Markov chains (Springer, Berlin). Kemeny, J.G. and J.L. Snell (1976), Finite Markov chains (Springer, New York). Kingman, J.C. (1963), The exponential decay of Markov transition probabilities, Proc. London Math. Sot'. 13, 337-358. Ledoux, J., G. Rubino and B. Sericola (1994), Exact aggregation of absorbing Markov processes using quasi-stationary distribution, J.

App. Probab. 31, 62~634. Pruitt, W.E. (1964), Eigenvalues of non-negative matrices, Ann. Math. Statist. 35, 1797-1800. Rosenblatt, M. (1971), Markov Processes. Structure and Asymptotic Behavior (Springer, New York). Rubino, G. and B. Sericola (1989), On weak lumpability in Markov chains, J. Appl. Probab. 26, 446457. Rubino, G. and B. Sericola (1991), A finite characterization of weak lumpable Markov processes. Part I: The discrete time case, Stochastic

Process Appl. 38, 195-204. Rubino, G. and B. Sericola (1993), A finite characterization of weak lumpable Markov processes Part II: The continuous time case,

Stochastic Process AppL 45, I 15 125. Seneta, E. (1981), Non-negative matrices and Markov chains (Springer, New York). Seneta, E. and D. Vere-Jones (1966), On quasi-stationary distributions in discrete-time Markov chains with a denumerable infinity of

states, ,L Appl. Probab. 3, 403~,34. Sumita, U. and M. Rieders (1989), Lumpability and time reversibility in the aggregation~lisaggregation method for large Markov chains,

Comm. Statist. Stochastic Models' 5, 63-81.