and markov with (a’m,n)-matrix l. abolnikov

Journal ofApplied Mathematics and Simulation Vol. 1, Issue 1, 1987.

NECESSARY AND SUFFICIENT CONDITIONS FOR THE ERGODICITY OFMARKOV CHAINS WITH TRANSITION Am,n (A’m,n)-MATRIX

L. ABOLNIKOVLoyola Marymount University

Los Angeles, California

and

A. DUKHOVNYScientific Research Institute on Automation

Odessa, U.S.S.R.

ABSTRACT

This paper isolates and studies a class of Markov chains with aspecial quasi-triangular form of the transition matrix [so-called Am,n(A’m,n)-matrix]. Many discrete stochastic processes encountered in

applications (queues, inventories and dams) have transition matriceswhich are special cases of a Am,n (A’m,n)-matrix. Necessary andsufficient conditions for the ergodicity of a Markov chain withtransition Am,n (A’m,n)-matrix are determined in the article in two

equivalent versions. According to the first version, these conditions areexpressed in terms of certain restrictions imposed on the generatingfunctions A (x) of the elements of the i-th row of the transition matrix,

0, 1, 2 in the other version they are connected with thecharacterization of the roots of a certain associated function in the unitcircle of the complex plane. Results obtained in the article generalize,complement, and refine similar results existing in the literature.

Key words: Markov chains, queues, inventories, dams, ergodicity,necessary condition, sufficient condition

AMS subject classification: 60J, 60K25

1. INTRODUCTION

A class of Markov stochastic processes with transition (or infinitesimal) matrix of the

14 JOURNAL OF APPLIED MATHEMATICS AND SIMULATION Vol. 1, Issue 1, 1987

(1.1) G

go,o go,1 go,2

gl,0 al,1 al,2

g2,0 g2,1 g2,2

gin,0 gin,1 gm,2

0 gm+l,1 gm+l,2

0 0 gm/2,2

(a "Am-matrix" or of the form G’ (a A’m-matrix*) was first investigated for the case offinite set of states by Abolnikov in 1]. A method was proposed for reducing the problemof finding the state probabilities for such a process to the simpler problem of finding thestate probabilities for an auxiliary process whose transition matrix is related to the Am(A’m)-matrix of the original process. (This was called, "the continuation method"). Inparticular, this method provides the solution for the finite dimensional case if a solution isavailable for the infinite-dimensional case. Markov chains with an infinite transition matrixwhich is a slightly modified analog of (1.1) was investigated by Neuts 11 ]. In this paper anew effective approach for finding the steady-state probabilities of the process wasproposed.

However, the questions related to the conditions for the ergodicity of suchprocesses were not considered in [1,11]. The present paper is devoted to this problem. Asthe main object of consideration we study a countable Markov chain whose transitionmatrix has form (1.1). Keeping in mind the specific features of processes which arefrequently encountered in application, we do not impose any restrictions on the transientprobabilities gij if < m j < n ), but suppose at the same time that the gij for > n > ndepend only on the difference of the indices and j. That is, the transition matrix ofprocesses considered in the paper has the following form:

*G’ is the transpose of G.

Necessary and Sufficient Conditions for the Ergodicity of Markov Chains:Abolnikov and Dukhovny 15

(1.2)

a00

al0

A= ano0

0

alo ao n-m ao n-m+l ao n-m+2

all al n-m al n-m+2 al n-m+2

anl an n-m an n-m+l an n-m+2

0 0 ko k

0 0 0 k0

or its transpose. Many stochastic processes encountered in applications have transitionmatrices which are special cases of A (or A’). As examples there are embedded Markovchains describing the behavior of a queue in the queueing systems M/G/l, G/M/1, G/M/nwith bulk arrivals, bulk service, with feedback, with a threshold, with "warmup", withswitchings, with hysteresis service, and with queue buildup; the state of a storage in thetheory of inventory control; the state of a dam in the Moran problem. In this papernecessary and sufficient conditions for the existence of the ergodic distribution of theprobabilities of states of a Markov chain with infinite transition Am,n (A’m,n-matrix) aredetermined. These conditions then are connected with the existence and the location of theroots of the function

EZm

k’ (z), k (z) k z.i=0

Some special cases (as a rule, connected with queueing systems) of this problem yieldingmostly sufficient conditions were discussed in Abolnikov/Dshalalow [2], Abolnikov [3],Bahary/Kolesar [4], Bailey [5], Delbrouck [6], Harris/Marlin [7], Kopocinska [8], andLoris-Teghem [9].

However, the arguments given suffer either from insufficient analytical justificationor from considerable restriction of the generality. The same can be said about thecharacterization of the roots of the function zrn- k (z)in the circle z < 1. InBahary/Kolesar [4] the question of determination of the roots of zm k (z) is left open. InHarris/Marlin [7] the proof of the statement about the number of the roots of the functionzrn k (z) is given only under the suggestion of the possibility of the analytical extension ofthe k (z) outside of the unit circle. The solution of the problem was obtained inAbolnikov/Dshalalow [2] and Abolnikov [3] only for the sufficient conditions and m 1.

2. DEFINITIONS AND TERMINOLOGY

Definition 1. We shall say that a finite (not necessarily square) or an infinite stochasticmatrix A aij is a Am,n-matrix (resp. A’m,n-matrix), n > m >_ 1, if aij 0 for > n and

> m (resp. aij 0 for > m and > m). If in addition aij kj_i+rn for > n, > m(resp. aij ki_j/m for > n, > m) then the matrix A is called homogeneous.


Another definition of a homogeneous Am,n-matrix (that is more convenient in thecase of infinite matrices) can be given in the following way. Let

A (z) aij zi,j=o

be a generating function of elements aij of the i-th row of a stochastic matrix A aij ),0, 1, 2 The set of functions A (z), 0, 1, 2, obviously, completely

determines the matrix A.

Definition 2. An infinite stochastic matrix A aij ), i, j 0, 1, 2 is called ahomogenous Am,n-matrix if

A (z) zi-m

r=OkrZ, i=n+ 1, n+ 2

Let r, r 0, 1, 2 be a homogeneous Markov chains with transition Am,n-matrix A aij ), i, j 0, 1, 2, We denote

(0)Pi-(r) p { r } ;/i Pi 0, 1, 2,

P(r) _(r) (r) _(r)=(P0 ’Pl ’P2 ),r=0, 1,2,...; zi(Z) gi

i=O

(r) rU (x) Pi X; W(X,Z)= Ui(X) Zi" Ixl<l Izl<l

i-0

U (X) U0 (X), U (x), U2 (x),(r)

p x;r=0

(r) Elim Pi Pi’ P (P0’ Pl’ P2 ) ;P(z) Pi z.r-, i=O

Let us point out some useful relationships among the vectors

p, P(r), u (x) and the matrix A, which will be used later.

First, since the chain r is homogeneous(2.1) pr+l pr AIf the chain r is ergodic then(2.2) P=PA,and, finally, it is easy to see that(2.3) u (x)= xu (x) A + p0.


The problem consists in finding necessary and sufficient conditions for the ergodicity of thechain r.

3. NECESSARY AND SUFFICIENT CONDITIONS FOR THE ERGODICITY OFMARKOV CHAINS WITH TRANSITION Am,n and A’m,n-MATRICES.

Theorem 1. A Markov chain r with transition irreducible and nonperiodic homogeneous

Am,n-matrix A =( aij ), i, 0, 1, 2,... is ergodic if and only if

(3 1) A’ (1)<oo i=0,1 2, n,

and(3.2) k’ (1) < m.Proof"Sufficiency. Let us put xj j, j 0, 1, 2,...; then it follows from (1.2) that

f A’ (1)-i, ifi= 0, 1, n

Z alj xj xj=0

k’ (1) m, if n+ 1,n+2,...

By the corollary of Moustafa [10] to Foster’s theorem we conclude that (3.1) and (3.2) are

sufficient for the ergodicity of the chain r.Necessity. Let us first prove the following lemma.Lemma. Consider a function

H (z) h z,i=0

which converges for z 1. Suppose that h < 0 if 0, 1, 2,..., k, and h > 0 if k+ 1,k+2,Assume further that H 0. If there exists z0 e (0,1), such that H (z0) > 0, the H (z) > 0

for any z e (z0, 1].Proof. Under the conditions of the lemma the following chain of inequalities for anyz e (z0, 1] is valid:

_z-(k+l) Z hizi<-z;(k+l) Z hizi0 _< -z;(k+l) Z hizi0 < z-(k+l) Z hi zii=0 i=0 i=k+l i=k+l

Since H (z) 0, at least one of the hi, 0, 1, 2,... is not equal to 0. Therefore, at leastone of the extreme inequalities in the chain is strict. The lemma then follows.

We now return to the proof of the theorem. If the chain {r is ergodic, then there exists astrictly positive vector P P0, Pl, P2, such that P PA.Multiplying this equation by the vector (0, 0, 0, zm, zre+l, with O’s in the first n + 1

places, where z [0,1] and taking into account (1.1) we obtain


ZH (z)

Pn+i+l mi=0 Z k (z)

where

n m-1 m-i-1

H(z) hi Zi zm Pi n+j+l- Pn+i+l hj.i=0 i=0 =0 i=0 j=0

It is easy to see that

(3.3) 0 < limH (z)

Pn+i+l < 1,zl- z

TMk (z) i-0

and since zm k (z)] z=l 0 then

(3.4) h (1)= lim H (z) 0.

It follows that H (z) < 0 for any z (0,1). Indeed, suppose that there exists z0 (0,1)such that H (z0) > 0. Then, since

m-1

H (0) h0 -k0

_Pn+i+l < 0

i-0

the Lemma implies h (1) > 0, which contradicts (3.4).

We shall need an additional property ofH (z), namely

(3.5) 0 < H’(1) <To prove this let us notice that k’ (1) < m, because otherwise there exists 5 > 0 such that

zm k (z) > 0 for any z e (1-5,1). Therefore,

H (z)< 0 in the same region, which is impossible because of (3.3).

mz -k(z)

Now to prove the right inequality in (3.5) it is enough to pass to the limits in (3.3) as z1-. Let us first consider the left inequality in (3.5) for m 1. It is obvious that ih > 0 for

any 0, 1, 2 On the other hand, since the chain r is irreducible, among thenumbers a n+j+l, 0, 1, 2, h, 0, 1, 2 so there exists at least one which isstrictly positive. It follows that there exists 0, such that

0 hi0 > 0 and, therefore, H’ (1) > 0 hi0 > 0.

If m > 1 thenm-2

H’(0) h -k1L Pn+i+l <0"i=0

In addition, since H (0) < 0 and H (1) 0, there exists z0 (0,1) such that H’ (z0) > 0. Bythe lemma it follows that H’ (1) > 0 and (3.5) is proved.


Now both of the statements (3.1) and (3.2) of the theorem 1 easily follow from(3.3) and (3.5). In fact, if we suppose that there exists i, such that A i’ (1) then, as canbe seen from the form of H (z), lim H’ (z) oo, which contradicts (3.5). Finally, thestatement (3.2) of the theorem can be obtained by passing to the limits in (3.3) as z -- 1-and taking into account (3.5). The theorem is proven.

Next we establish an analogous statement for a Markov chain r, r 0, 1, 2,with transition matrix of A’m,n-type.

Theorem 2: A Markov chain r, r 0, 1, 2, with transition irreducible aperiodic

homogeneous A’m,n-matrix A aij ), i, j 0, 1, 2, is ergodic if and only if

(3.6) k’ (1) > m

Proof"Sufficiency.Let us put x max {j-n,0} 0, oo. Then taking into account (1.2) it is easy to show that

0 ,ifi <n- m

a x x Di-n/m k (z) zif > n m

-0 z )z(1-z

where Dix is an operator defined by the relation

(3.7)1 )iF (x,y)Dix F (x,y)

Ox x=0

Note that DK (z)- z

TM j0 kj _> 0,

z 1-z ’j=0

and, therefore, if < m,m

(3.8) D k(z)-zm, iz k()-z _>0,z )2 z1-z j=0

and if > m 1m

Dk (z)- z

m

(3.9) Di+1 k(z)’- z_<

z )2 z )2(1-z (1-zHence, the sequence

ifi<m

k.- 1 <O, ifi> m


ri k (z) zTM

(l-z)2

i=1

either has a finite limit as --) ,,, or tends to +,. If k’(1) > m then using a Tauberiantheorem in the case of the finite limit we obtain:

rn Ill

lim Dk (z) z

limk (z) z

-k’ (1) + m < 0z 2 1-zi---), 1- z ) z-l-0

so that in this case there exist s and M < 0 such that

Z x.-x.<M, ifi>s.aijj=0

Again appealing to the criterion of Moustafa 10] we conclude that (3.6) is sufficient for theergodicity of the chain {r.Necessity. If the chain r is ergodic then there exists a strictly positive probability vector P

(P0, Pl, P2, such that P PA. Multiplying this equation by the vector (x 0, xl, x2,), where xj max {j-n,0}, j 0, 1, 2 after some simple transformations, we

obtain

rn

Dk(z)- z

(3.10) Pn+l-m+l zi=0 (l-z)

2

Suppose that k’(1) < m. Then

rn rn

limDk(z)- z

lim *’-’__ --Dzk(z)-z

k(l)+m>0.z )2 1 zioo 1 z i---)oo j=0

Together with (3.8) and (3.9), this implies thatrn

D .k(z)-z >0_z (1_z)2

for all 0, 1, 2,..., which contradicts (3.10). The theorem is proved.

4. ERGODICITY OF THE MARKOV CHAIN WITH TRANSITION Am,n(A’m,n)-MATRIX

AND CHARACTERIZATION OF THE ROOTS OF THE FUNCTION zm k (z).

The conditions of ergodicity proved in section 3 are closely connected with thenumber and location of the roots of the function z rn k (z). More precisely, the followingtheorem is valid.

Theorem 3


A. If k’(1) < m, then the function zm k (z) has exactly m roots (countingmultiplicities) in the disc z < 1. The roots lying on the boundary are simple and,for some integer r, they are all r-th roots of 1.

g. If k’(1) > m, then the function zm k (z) has exactly m roots (countingmultiplicities) in the open disc z I< 1; on the boundary there can be r additionalsimple roots which are all the r-th roots of 1, where r is an integer, 1 < r _< m.

Proof: We will need the following result.Lemma. The function zm k (z) has roots on the unit circle z 1 if and only if there

exists r such that m r and for any such that k 0, r. If these conditions are satisfied,

then all roots of this kind coincide with the roots of the equation zr 1 0, where r is themaximal number having this property.

Proof: Suppose that z0 is a root of the equation zm k (z) 0 such that z01 1. Then

k (z0)I- zl- 1.

On the other hand,

Ik(z0) I= IE kizi01 < kilz01i=li=0 i=0

and the equality is attained if and only if

mz0 Iz01 lforanyi, suchthatkl 0andz0 1.

The realization of both of these conditions simultaneously is possible only if the conditionsof the Lemma are satisfied. In this case, obviously,

z0 1.

Otherwise, if the conditions of the Lemma are satisfied then for any root z0 of the

equation zr 1 0 we obtainrn

k(z0) 1 z0,

which implies that the root z0 is a root of the equation k (z) zm 0. The Lemma isproved.

Now we return to the proof of the theorem.

A. First we suppose that the number r, appearing in the Lemma, is 1. Then thefunction zm k(z) has only 1 root on the unit circle. This root is equal to 1 and is simple,since k’(1) m’. We will prove that in this case zm k (z) has exactly m- 1 roots inside F:Iz 1. Consider an auxiliary function

1- z-mk (z)f (z)

-11-z


Clearly, f (z) 0 for all z, zl 1 since the numerator of f (z) may be zero only ifz 1, but f (1) m -k’(1) > 0. Suppose now that Indr f (z) denotes the difference between

the number of the roots and the number of the poles of the function f (z) inside F.

By the principle of the argument:

[ ] (),(4.1) Indrf(Z -.A Argf(z) Ind 1-z-ink(z) -Ind 1-z-1

where Ar Arg f (z) is the increment of the argument of f (z) when the argument of z ei

increases from 0 to 2ft.

Let us consider the right side of (4.1). Since Arg f (1) 0, it is easy to notice that

[ (ei)] lim [1 e-i] =+lim Arg 1 e-imk(I)-)_+O --->+0 "’"At the same time if

-imk ( )(0,2), e eia’

< 1 and, hence,

It follows that

z-1Indr[1-z-mk(z)] =-= Indr[ 1- ]and, therefore,

From the fact that

Indrf(Z) 0

f (z)rnz k (z)rrrl

z (z-l)

has exactly m 1 poles inside F, we conclude that the number of roots of f (z) inside F isalso m- 1.

Thus, the total number of roots (together with z 1) of the function zm k (z) is theunit disc z < is equal to m.

(4.2)

Now let us consider the general case r > 1. We introduce the function

m

F(z) z k.zi=0

By the definition of r it follows that all exponents of z in (4.2) are integers.Applying the previous reasonings to (4.2) we obtain that f (z) has exactly


m1 roots in the region zl < i and one root z 1 on the boundary.

The set of all r-th roots of the roots of F (z) gives us all roots of zm k (z). On the otherhand, it is obvious, that any root of zm k (z) raised to the r-th power is a root of F (z).Therefore, the set of all roots of zm k (z) is described completely. The number of them is

m; m r are inside the region z <- 1, and r roots are on the boundary.

Suppose now that z0 is a root of zm k (z) such that z0l 1. Then:

ik,(z0) l=li ikizi-ll= iliki ikiZo, -< i=l

k’(1)_Z =lk’(1) l<rn

It follows that k’(z0) e mzm-l, which implies that all roots on the boundary are simple. PartA of the theorem is proved.

Next we shall prove the part B. Since k’(1) > m, there exists > 0 such that

k(p) < pm for any p e [1 8,1]. Therefore, the inequality k (z) <1 zrn is correct for

any z, such that zl p. Using the theorem of Rouche, we obtain that zTM k (z) has

exactly m roots in the region zl < 1. The number of roots on the boundary, as before,depends on the value of r. If r > 1 then, by the lemma, there are r roots on the boundarywhich represent a group of r-th roots of 1. The simplicity of these roots is proved asbefore.

REFERENCES

[1]

[2]

[3]

[41

[5]

[61

[7]

Abolnikov, L.M. Investigation of a Class of Discrete Markov Processes. Izv.ANSSR, Tekhnicheskaya Kibernetika, N2:69-82 (1977). (English translation:Engineering Cybernetics, 15(1977)N2:51-63).

Abolnikov, L.M., Dshalalow, E. A. Feedback Queueing Systems: Duality Principleand Optimization. Automation and Remote Control 39:11-20 (1978).

Abolnikov, L. M. A Single Level Control in Moran’s Problem with Feedback.Operations Research Letters 2, N1:16-19 (1983).

Bahary, E., Kolesar, P. Multilevel Bulk Service Queues. Oper. Res. 20:407-420(1972).

Bailey, N. T. On Queueing Process with Bulk Service. J. Roy. Statist. Soc. B.16:80-87 (1954).

Delbrouek, L. A Feedback Queueing System with Batch Arrivals, Bulk Service,and Queue-Dependent Service Time. J. Assoc. Comput. Mach. 17:314-323 (1970).

Harris, C. M., Marlin, P. E. A note on Feedback Queues with Bulk Service. J.Assoc. Comput. Mach. 19:727-733 (1972).


[8]

[9]

[10]

[11]

Kopocinska, I. GI/M/I Queueing Systems with Service Rates Depending on theLength of Queue. Zastosow. Matem. 11:265-279 (1970).

Loris-Teghem, J. Condition Necessaire d’Ergodisme pour un ProcessusStochastique lie a un Systeme d’Attente a Arrivees et Services en Groupesd’Effectif Aleatore. Bull. C1. Sci. Acad. Roy. Belg. 52:382-389 (1966).

Moustafa, M.D. Input-Output Markov Processes. Proc. Koninkijke NederlandeAkad. Wetenschappen 60:112-118 (1957).

Neuts, M. F. Queues Solvable without Rouche’s Theorem. Operat. Res.27(4):767-781 (1979).

Submit your manuscripts athttp://www.hindawi.com

Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

MathematicsJournal of


Mathematical Problems in Engineering

Hindawi Publishing Corporationhttp://www.hindawi.com

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of


Probability and StatisticsHindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

Journal of


Mathematical PhysicsAdvances in

Complex AnalysisJournal of


OptimizationJournal of


CombinatoricsHindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

International Journal of


Operations ResearchAdvances in

Journal of


Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

International Journal of Mathematics and Mathematical Sciences


The Scientific World JournalHindawi Publishing Corporation http://www.hindawi.com Volume 2014


Algebra

Discrete Dynamics in Nature and Society



Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttp://www.hindawi.com

Volume 2014 Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

Stochastic AnalysisInternational Journal of

and markov with (a’m,n)-matrix l. abolnikov

Documents