This article was downloaded by: [Bibliothèques de l'Université de Montréal]On: 02 December 2014, At: 14:01Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,37-41 Mortimer Street, London W1T 3JH, UK

Stochastic Analysis and ApplicationsPublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/lsaa20

Stochastic Processes with Age-Dependent TransitionRatesMrinal K. Ghosh a & Subhamay Saha aa Department of Mathematics , Indian Institute of Science , Bangalore, IndiaPublished online: 29 Apr 2011.

To cite this article: Mrinal K. Ghosh & Subhamay Saha (2011) Stochastic Processes with Age-Dependent Transition Rates,Stochastic Analysis and Applications, 29:3, 511-522, DOI: 10.1080/07362994.2011.564455

To link to this article: http://dx.doi.org/10.1080/07362994.2011.564455

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) containedin the publications on our platform. However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of theContent. Any opinions and views expressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon andshould be independently verified with primary sources of information. Taylor and Francis shall not be liable forany losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use ofthe Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematicreproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Stochastic Analysis and Applications, 29: 511–522, 2011Copyright © Taylor & Francis Group, LLCISSN 0736-2994 print/1532-9356 onlineDOI: 10.1080/07362994.2011.564455

Stochastic Processes with Age-DependentTransition Rates

MRINAL K. GHOSH AND SUBHAMAY SAHA

Department of Mathematics, Indian Institute of Science,Bangalore, India

We study stochastic processes with age-dependent transition rates. A typical exampleof such a process is a semi-Markov process which is completely determined bythe holding time distributions in each state and the transition probabilities ofthe embedded Markov chain. The process we construct generalizes semi-Markovprocesses. One important feature of this process is that unlike semi-Markovprocesses the transition probabilities of this process are age-dependent. Undercertain condition we establish the Feller property of the process. Finally, wecompute the limiting distribution of the process.

Keywords Feller property; Semi-Markov processes; Transition probabilities;Transition rate function.

Mathematics Subject Classification 60J27; 60J75.

1. Introduction

We construct a new class of stochastic processes taking values in a countablestate space S = �0� 1� 2� � � � �. This new class of processes subsumes semi-Markovprocesses. A semi-Markov process �Xt� taking values in S is determined by aninfinite matrix �pij� with pii = 0 for each i ∈ S and a family of distribution functionsF�� � i�, i ∈ S. The matrix �pij� determines the transition probabilities, that is, ifXt = i then the next state of the process is j with probability pij . The functionF�� � i� gives the distribution of the sojourn (or holding or waiting) time in state i.If the waiting time distribution at each state i is exponential then �Xt� is a Markovprocess; otherwise �Xt� is not Markov. If at time t, Yt denotes the age of Xt, (i.e.,if Xt = i then Yt denotes the time Xt has already spent in i) then ��Xt� Yt�� is aMarkov process [2]. For each i ∈ S, let F�� � i� have a positive density denoted by

Received June 26, 2010; Accepted July 2, 2010This work is supported in part by a grant from UGC via DSA-SAP-Phase IV and in

part by SPM fellowship of CSIR.The authors wish to thank Manjunath Krishnapur for helpful discussions.Address correspondence to Mrinal K. Ghosh, Department of Mathematics, Indian

Institute of Science, Bangalore 12, India; E-mail: mkg@math.iisc.ernet.in

511

Dow

nloa

ded

by [

Bib

lioth

èque

s de

l'U

nive

rsité

de

Mon

tréa

l] a

t 14:

01 0

2 D

ecem

ber

2014

512 Ghosh and Saha

f�� � i�. For i �= j, let

�ij�y� �=pijf�y � i�1− F�y � i� � (1.1)

Then from the properties of semi-Markov processes it is easy to see that [1] for i �= j

��Xt+h = j �Xt = i� Yt = y� = �ij�y�h+ o�h�� (1.2)

Thus, if the age is y, that is, Yt = y, then �ij�y� gives the infinitesimal jump rates of�Xt�. If �ij�y� is independent of y, then Xt is a Markov chain with its Q-matrix ��ij�.This happens if and only if F�� � i� ∼ Exp��i�, �i > 0.

In this article, we consider 1.2 as the starting point in the construction of theprocess �Xt�. That is we assume that for i� j ∈ S� i �= j, �ij � 0��� → 0��� aregiven functions. We construct the process ��Xt� Yt�� which satisfies 1.2. We showthat the process thus constructed generalizes semi-Markov processes.

The rest of the article is structured as follows. Section 2 describes theconstruction of ��Xt� Yt�� satisfying 1.2. This is achieved by an explicit constructioninvolving an appropriate Poisson random measure. Section 3 deals with the Fellerproperty of ��Xt� Yt��. The limiting distributions of ��Xt� Yt�� are treated in Section 4.

2. Construction of the Process

Let ��� ��� be the underlying probability space and S = �0� 1� 2� � � � �. For i� j ∈S� i �= j and y ∈ �+ �=0����, let

�ij � 0��� → 0���

be given measurable functions. Set

�ii�y� = −∑j �=i

�ij�y��

We make the following assumption, which is in force throughout this article:

(A1) supi∈S�y>0�−�ii�y�� < �.

For i �= j and y ∈ �+, let �ij�y� be consecutive (with respect to lexicographicordering) right open, left closed intervals of the real line of length �ij�y�.

We define a function h � S ×�+ ×� → � by

h�i� y� z� ={j − i if z ∈ �ij�y�

0 otherwise�(2.1)

We also define a function g � S ×�+ ×� → � by

g�i� y� z� =y if z ∈⋃

j

�ij�y�

0 otherwise�(2.2)

Dow

nloa

ded

by [

Bib

lioth

èque

s de

l'U

nive

rsité

de

Mon

tréa

l] a

t 14:

01 0

2 D

ecem

ber

2014

Processes with Age-Dependent Transition Rates 513

Now let Xt = X0 +

∫ t

0

∫�h�Xs−� Ys−� z�℘�ds� dz�

Yt = Y0 + t −∫ t

0

∫�g�Xs−� Ys−� z�℘�ds� dz��

(2.3)

where ℘�ds� dz� is a Poisson random measure on �+ ×� with intensity measureds × dz, the product Lebesgue measure on �+ ×� and the integrals are over �0� t .From the results in [3, Chap. IV, p. 231] we know that there exists an a.s uniquestrong solution of (2.3) and ��Xt� Yt�� is a strong Markov process.

We now calculate the generator of the process ��Xt� Yt��. Let f � S ×�+ → �be continuously differentiable in the second variable. Then applying Itô’s formulato f we get

f�Xt� Yt�− f�X0� Y0� =∫ t

0

�f

�y�Xs� Ys�ds +

∫ t

0

∫�f�Xs− + h�Xs−� Ys−� z��

Ys− + g�Xs−� Ys−� z��− f�Xs−� Ys−� ℘�ds� dz�

=∫ t

0

�f

�y�Xs� Ys�ds +

∫ t

0

∫�f�Xs− + h�Xs−� Ys−� z��

Ys− + g�Xs−� Ys−� z��− f�Xs−� Ys−� ℘̃�ds� dz�

+∫ t

0

∫�f�Xs− + h�Xs−� Ys−� z�� Ys− + g�Xs−� Ys−� z��

− f�Xs−� Ys−� ds × dz

=∫ t

0

�f

�y�Xs� Ys�ds +

∫ t

0

∫�f�Xs− + h�Xs−� Ys−� z��

Ys− + g�Xs−� Ys−� z��− f�Xs−� Ys−� ℘̃�ds� dz�

+∫ t

0

∑j �=Xs−

�Xs−j�Ys−�f�j� 0�− f�Xs−� Ys−� ds�

where ℘̃ is the martingale associated with ℘.Therefore, the generator of the process ��Xt� Yt�� denoted by � is given by

�f�i� y� = �f

�y�i� y�+∑

j �=i

�ij�y�f�j� 0�− f�i� y� � (2.4)

From (2.3) it is clear that �Xt� is a right-continuous (since the integration is over�0� t ) jump process taking values in S. For a fixed � ∈ , Xt��� has a jump at t0to the state j if and only if

℘(�t0�×�Xt0−���j

�Yt0−����)��� �= 0�

Evidently Xt and Yt jump at the same time and Yt0��� = 0 if Xt��� has a jump at t0.Let Tn denote the nth jump time of Xt. Let

N�t� = max�n � Tn ≤ t�

Dow

nloa

ded

by [

Bib

lioth

èque

s de

l'U

nive

rsité

de

Mon

tréa

l] a

t 14:

01 0

2 D

ecem

ber

2014

514 Ghosh and Saha

be the total number of jumps up to time t. Thus,

TN�t� ≤ t ≤ TN�t�+1

and

Yt = t − TN�t��

Now using the property of Poisson random measure

��T1 > y �X0 = i� Y0 = 0� = exp(−∫ y

0

∑j �=i

�ij�u�du

)�

Therefore,

F̃ �y � i� �= ��T1 ≤ y �X0 = i� Y0 = 0�

= 1− exp(−∫ y

0

∑j �=i

�ij�u�du

)� (2.5)

Thus, F̃ �� � i� is the waiting time distribution in the state i.Again,

p̂ij �= ��XTn+1= j �XTn

= i�

=∫ �

0��XTn+1

= j �XTn= i� Tn+1 = t�dF̃ �t � i�

=∫ �

0

�ij�t�∑j �=i �ij�t�

dF̃ �t � i��

If we define �n = XTn, then ��n� is a discrete time Markov chain with transition

probabilities

p̂ij =∫ �

0

�ij�t�∑j �=i �ij�t�

dF̃ �t � i�� (2.6)

We now give a few examples of processes of the above type.

Example 1. For i �= j, let the transition functions �ij’s be constants, that is

�ij�y� = qij ∀y ≥ 0

for some non-negative constants qij . In this case �Xt� is a continuous time Markovchain with rate matrix �qij�.

Example 2. Let �pij� be an (infinite) transition probability matrix with pii = 0 forall i and F�� � i�� i ∈ S a family of distribution functions with densities f�� � i�. Let

�ij�y� = pij

f�y � i�1− F�y � i� �

Dow

nloa

ded

by [

Bib

lioth

èque

s de

l'U

nive

rsité

de

Mon

tréa

l] a

t 14:

01 0

2 D

ecem

ber

2014

Processes with Age-Dependent Transition Rates 515

Then it can be shown that [1], �Xt� is a semi-Markov process with waiting timedistributions F�� � i� and transition probabilities pij ; �Yt� is the associated age process.

The above two examples show that our process subsumes both Markov andsemi-Markov processes. Now we give an example that is neither a Markov nor asemi-Markov process in the conventional sense.

Example 3. Suppose a financial market can be in three states, namely, up state,down state, and moderate state. If the down state is denoted by 0, the moderatestate by 1 and the upstate by 2, then the age-dependent transition rates are given asfollows:

�01�y� =

34

if y ≤ 1

14

if y > 1

� �02�y� =

34

if y ≤ 1

14

if y > 1

�

�10�y� =12

∀y ≥ 0� �12�y� =12

∀y ≥ 0�

�20�y� =

25

if y ≤ 1

35

if y > 1

� �21�y� =

35

if y ≤ 1

25

if y > 1

�

In this process, unlike Markov and semi-Markov processes, transitionprobabilities are not independent of the transition times. On the contrary the age-dependent transition probabilities pij�y� �= ��XTn+1

= j �XTn= i� Tn = y� = �ij �y�∑

k �=i �ik�y�

for this process are given by

p01�y� =

34

if y ≤ 1

14

if y > 1

� p02�y� =

34

if y ≤ 1

14

if y > 1

�

p10�y� =12

∀y ≥ 0� p12�y� =12

∀y ≥ 0�

p20�y� =

25

if y ≤ 1

35

if y > 1

� p21�y� =

35

if y ≤ 1

25

if y > 1

�

Thus, it is clear that the process described in this article is not a conventional semi-Markov process, but a more general one that generalizes semi-Markov processes.

Remark 2.1. Note that the transition probabilities �p̂ij� given by (2.6) and thewaiting time distributions F̃ �� � i� given by (2.5) determine a semi-Markov process�X̃t�. It is easy to see that the semi-Markov process �X̃t�, thus, determined isdifferent from the process �Xt� determined by (2.3). Thus a semi-Markov process isalways embedded in the more general process �Xt�.

Dow

nloa

ded

by [

Bib

lioth

èque

s de

l'U

nive

rsité

de

Mon

tréa

l] a

t 14:

01 0

2 D

ecem

ber

2014

516 Ghosh and Saha

3. Feller Property

From now on we make the following additional assumption on �ij�y�:

(A2) For all i� j ∈ S, �ij�y� is continuous in y.

To establish the Feller property of ��Xt� Yt�� we first compute its transitionprobability function P�t� �i� s�� �j��� where � be a Borel subset of �+. If ��t�

denotes the natural filtration of ��Xt� Yt�� and � denotes the first jump time after sthen

P�t� �i� s�� �j��� = �i�sI�j���Xt� Yt�

= �i�sI�j�×��Xt� Yt�I�� > t�+�i�sI�j�×��Xt� Yt�I�� ≤ t�

= �i�sI�j�×��Xt� Yt�I�� > t�+�i�sI�� ≤ t���I�j�×��Xt� Yt� ����

= exp(−∫ s+t

s

∑k �=i

�ik�u�du

)I�j��i�I��s + t�

+∑j �=i

∫ t

0exp

(−∫ u

0

∑k �=i

�ik�s + y�dy

)× �ij�s + u�P�t − u� �j� 0�� �j�×��du�

From the expression above, it follows that the Markov semigroup �Tt� for theprocess ��Xt� Yt�� satisfies

Ttf�i� s� �= �i�sf�Xt� Yt�

= exp(−∫ s+t

s

∑k �=i

�ik�u�du

)f�i� s + t�

+∑j �=i

∫ t

0exp

(−∫ u

0

∑k �=i

�ik�s + y�dy

)�ij�s + u�Tt−uf�j� 0�du� (3.1)

where f � S ×�+ → � is a bounded measurable function.Now if f�i� s� is continuous in s, it then follows that Ttf�i� s� → f�i� s� as t ↓ 0.

Therefore for any f�i� s� continuous in s, Ttf�i� s� is continuous from the right withrespect to t. We now show that Ttf�i� s� is continuous in s. The first summand inthe right hand side of (3.1) has this property. To prove that the second summandhas this property it is sufficient to show that for all j �= i,

∫ t

0exp(−∫ u

0

∑k �=i

�ik�s + y�dy

)�ij�s + u�Tt−uf�j� 0�du (3.2)

is continuous in s. Under (A2), since the measures

�s��� =∫�exp(−∫ u

0

∑k �=i

�ik�s + y�dy

)�ij�s + u�du

Dow

nloa

ded

by [

Bib

lioth

èque

s de

l'U

nive

rsité

de

Mon

tréa

l] a

t 14:

01 0

2 D

ecem

ber

2014

Processes with Age-Dependent Transition Rates 517

are weakly continuous in s, it follows that

lims→s0

∫g�u��s�du� =

∫g�u��s0

�du� (3.3)

for any bounded measurable function whose set of discontinuity points is of�s0

-measure zero. Since Tt−uf�j� 0� is continuous from the left in u, the set ofdiscontinuities of this function is at most countable. However the measure �s0

iscontinuous so that any countable set has measure zero. Thus (3.2) follows from (3.3)with g�u� = Tt−uf�j� 0�. We have thus proved the following theorem:

Theorem 3.1. Under (A1) and (A2), ��Xt� Yt�� is a Feller process.

4. Limiting Distributions

We study the long term behavior of the process ��Xt� Yt��. For the sake of simplicitywe assume (A2). This assumption can be dropped as we comment later. First weneed some definitions. Define for all i� j and t ≥ 0

Gij�t� = ��Nj�t� > 0 �X0 = i� Y0 = 0�

where Nj�t� is the number of transitions into state j up to time t. Thus, Gij�·� is thedistribution of the time until the first transition into state j given the process startsin i.

We also define

F̃ij�t� = ��T1 ≤ t �X0 = i� Y0 = 0� XT1= j�

= 1p̂ij

∫ t

0exp(−∫ s

0

∑k �=i

�ik�u�du

)�ij�s�ds�

The quantity F̃ij�·� is the distribution of the time until the next transition given theprocess has just entered i and will make the next transition into state j.

We also define the following moments

�ij =∫ �

0tdGij�t�� �ij�t� =

∫ �

0tdF̃ij�t�

�i =∫ �

0tdF̃ �t � i� =

∫ �

0exp

(−∫ t

0

∑j �=i

�ij�u�du

)dt�

We now recall the familiar notions of irreducibility, recurrence etc. for theprocess �Xt�.

Definition 4.1. Let i� j ∈ S,

(i) states i and j are said to communicate if either i = j or

Gij���Gji��� > 0�

(ii) the process is said to be irreducible if all the states communicate with eachother;

Dow

nloa

ded

by [

Bib

lioth

èque

s de

l'U

nive

rsité

de

Mon

tréa

l] a

t 14:

01 0

2 D

ecem

ber

2014

518 Ghosh and Saha

(iii) a state i is said to be recurrent if Gii��� = 1;(iv) a recurrent state i is said to be positive recurrent if �jj < �;(v) if all states are positive recurrent then the process is said to be positive

recurrent.

Note that

Gij��� = ��XTn= j for some n > 0 �X0 = 0� Y0 = 0��

Thus for �Xt�, a state i is recurrent (communicates with j) if and only if, the state iis recurrent (communicates with j) in the embedded Markov chain �XTn

� which hastransition probabilities p̂ij given by (2.6).

We now want to compute

limt→���Xt = j �X0 = i� Y0 = y��

If �Xt� is recurrent then if we consider the process starting from j with age 0, thenthe process is a regenerative process with time of regeneration being the first time itcomes back to j again. Using this fact we have the following theorem:

Theorem 4.1. Assume (A1) and (A2). If i communicates with j, Gjj�·� is non-lattice(i.e., it is not supported on a lattice of the form a+ bn for some a� b �= 0 and n ∈ �)and �jj < � then

pj = limt→���Xt = j �X0 = i� Y0 = y� = �j

�jj

� (4.1)

Proof. From the theory of delayed regenerative processes [4, chap. 5]

pj =expected amount of time spent in state j during one cycle

expected length of the cycle

= �j

�jj

�

It is easy to see that for all i� j

�ij = �i +∑k �=j

pik�kj� (4.2)

Now we make the following assumption.

(A3): The process �Xt� is irreducible, positive recurrent and non-lattice. Also theembedded Markov chain is positive recurrent with invariant distribution ��j�.

Theorem 4.2. Under (A1), (A2), and (A3), for j ∈ S

�jj�j =�∑i=0

�i�i

Dow

nloa

ded

by [

Bib

lioth

èque

s de

l'U

nive

rsité

de

Mon

tréa

l] a

t 14:

01 0

2 D

ecem

ber

2014

Processes with Age-Dependent Transition Rates 519

and

pj =�j�j∑�i=0 �i�i

�

Proof. Multiplying both sides of (4.2) by �i and summing up we get,

�∑i=0

�i�ij =�∑i=0

�i�i +∑k �=j

�kj

�∑i=0

�ipik

=�∑i=0

�i�i +∑k �=j

�k�kj�

This implies

�j�jj =�∑i=0

�i�i�

The second assertion follows. �

Next we want to find

limt→���Xt = j� XN�t�+1 = k� Zt ≤ x �X0 = i� Y0 = y��

where Zt denotes the residual life at time t. To this end we first consider,

R�t� = ��Xt = j� XN�t�+1 = k� Zt ≤ x �X0 = i� Y0 = 0��

Suppose j is recurrent, then conditioning on T , the time of first return to state j,we get

R�t� =∫ t

0R�t − s�dGjj�s�+ h�t� (4.3)

where

h�t� =∫ �

t��Xt = j� XN�t�+1 = k� Zt ≤ x �X0 = j� Y0 = 0� T = s�dGjj�s�

=∫ �

t��t < T1 ≤ t + x�XN�t�+1 = k �X0 = j� Y0 = 0� T = s�dGjj�s�

=∫ �

0��t < T1 ≤ t + x�XN�t�+1 = k �X0 = j� Y0 = 0� T = s�dGjj�s�

since T1 ≤ T . Therefore,

h�t� = ��t < T1 ≤ t + x�XN�t�+1 = k �X0 = j� Y0 = 0��

Equation (4.3) is a renewal type equation and so its solution is given by [4, chap. 3]

R�t� = h�t�+∫ t

0h�t − s�dm�s��

where m�s� =∑�n=0�Gjj�n�s�, �Gjj�n�s� denotes n-times convolution.

Dow

nloa

ded

by [

Bib

lioth

èque

s de

l'U

nive

rsité

de

Mon

tréa

l] a

t 14:

01 0

2 D

ecem

ber

2014

520 Ghosh and Saha

Thus, by key-renewal theorem [4, chap. 3, p. 42],

limt→�R�t� = 1

�jj

∫ �

0h�t�dt

= 1�jj

∫ �

0��t < T1 ≤ t + x�XN�t�+1 = k �X0 = j� Y0 = 0�dt

= 1�jj

∫ �

0

(∫ t+x

xexp

(−∫ y

0

∑l �=j

�jl�u�du

)�jk�y�dy

)dt�

Since every step above goes through for delayed renewal processes as well, we haveestablished the following result:

Theorem 4.3. Under (A1)–(A3)

limt→���Xt = j� XN�t�+1 = k� Zt ≤ x �X0 = i� Y0 = y�

= 1�jj

∫ �

0

(∫ t+x

xexp

(−∫ y

0

∑l �=j

�jl�u�du

)�jk�y�dy

)dt� (4.4)

Note that

Yt > x ⇐⇒ no transitions in t − x� t

⇐⇒ Zt−x > x�

Using the above observation we have the following corollary:

Corollary 4.1. Under (A1)–(A3)

limt→���Xt = j� Yt ≤ x �X0 = i� Y0 = y� = 1

�jj

∫ x

0exp

(−∫ t

0

∑l �=j

�jl�u�du

)dt� (4.5)

Remark 4.1. Note that (A2) is only necessary for proving the Feller property.Other than that we can do away with this assumption. As a consequence of (A2) F̃has a continuous density which is not required but only makes the expressions nicer.In fact for continuous density we do not need the �ij’s to be individually continuousbut we only need

∑j �=i �ij to be continuous. This is the case in Example 3.

Example 3 continued. Here, we calculate various quantities for the processdescribed in Example 3. First we calculate the waiting time distributions F̃ �· � i�.

F̃ �y � 0� = 1− exp

(−∫ y

0

∑j=1�2

�0j�u�du

)= 1− e−y�

Dow

nloa

ded

by [

Bib

lioth

èque

s de

l'U

nive

rsité

de

Mon

tréa

l] a

t 14:

01 0

2 D

ecem

ber

2014

Processes with Age-Dependent Transition Rates 521

Similarly for i = 1� 2. Thus the waiting time distributions are exponential withparameter 1. Hence,

�i = 1 for i = 0� 1� 2�

Transition probabilities of the embedded Markov chain are given by

p̂01 =∫ �

0exp

(−∫ t

0

∑j=1�2

�0j�y�dy

)�01�t�dt

=∫ 1

0

34e−tdt +

∫ �

1

14e−tdt

= 34− 1

2e−1�

By similar calculations we get the following transition probability matrix for theembedded chain

0 34 − 1

2e−1 1

4 + 12e

−1

12 0 1

2

35 − 1

5e−1 2

5 + 15e

−1 0

So if ��i� is the invariant distribution of this chain then it is given by the followingsystem of equations:

�0 =12�1 +

(35− 1

5e−1

)�2

�1 =(34− 1

2e−1

)�0 +

(25+ 1

5e−1

)�2

�2 =(14+ 1

2e−1

)�0 +

12�1

�0 + �1 + �2 = 1

Solving the above equations we get approximately,

�0 = 0�3390� �1 = 0�3427� �2 = 0�3185�

Therefore,

p0 =�0�0∑i �i�i

= 0�3390�

Similarly,

p1 = 0�3427� p2 = 0�3185�

Now we calculate,

limt→���Xt = j� XTN�t�+1

= k� Zt ≤ x �X0 = i� Y0 = y�

Dow

nloa

ded

by [

Bib

lioth

èque

s de

l'U

nive

rsité

de

Mon

tréa

l] a

t 14:

01 0

2 D

ecem

ber

2014

522 Ghosh and Saha

By theorem (4.3) we have,

limt→���Xt = 2� XTN�t�+1

= 0� Zt ≤ x �X0 = i� Y0 = y�

= 1�22

∫ �

0

(∫ t+x

texp

(−∫ y

0

∑l=0�1

�2l�u�du

)�20�y�dy

)dt�

Suppose x ≤ 1 then we have,

1�22

∫ �

0

(∫ t+x

texp

(−∫ y

0

∑l=0�1

�2l�u�du

)�20�y�dy

)dt

= P2

�2

[∫ 1−x

0

(25

∫ t+x

te−ydy

)dt +

∫ 1

1−x

(25

∫ 1

te−ydy + 3

5

∫ t+x

1e−ydy

)dt

+ 35

∫ �

1

(∫ t+x

te−ydy

)dt

]= 0�3185×

(25− e−x + 1

5xe−x

)�

Similar calculations can be done for other j and k and also for x > 1.

References

1. Ghosh, M.K., and Goswami, A. 2009. Risk minimising option pricing in a semi-Markovmodulated market. SIAM J. Control Optim. 48:1519–1541.

2. Gikhman, I.I., and Skorokhod, A.V. 2004. The Theory of Stochastic Processes II. Classicsin Mathematics, Springer-Verlag, Berlin.

3. Ikeda, N., and Watanabe, S. 1989. Stochastic Differential Equations and DiffusionProcesses, North Holland, Amsterdam.

4. Ross, S.M. 1992. Applied Probability Models with Optimization Applications, Dover,New York.

Dow

nloa

ded

by [

Bib

lioth

èque

s de

l'U

nive

rsité

de

Mon

tréa

l] a

t 14:

01 0

2 D

ecem

ber

2014