A STOCHASTIC CONSTRAINED OPTIMAL REPLACEMENT MODEL*

Peter J. Kalman

Department of Economics State University of New York

Stony Brook, N.Y.

ABSTRACT

In this paper a stochastically constrained replacement model is formulated. This model determines a sequence of replacement dates such that the total “current account” cost of all future costs and capital expenditures over an infinite time horizon for the n initial incum- bent machines is minimized subject to the constraints that an expected number of machines are in a chosen utility class at any point in time. We then indicate one possible solution method for the model.

I. INTRODUCTION The paper is structured as follows. We first define the basic notation in section 2. Next, in section 3,

an analytical model is developed which determines a sequence of replacement dates for the ith initial incumbent machine ( i = 1, 2, . . ., n ) such that the total “current account” cost of all future costs and capital expenditures over an infinite time horizon is minimized subject to the constraint that at any point in time there exists a desired expected number of machines in any chosen “utility” class. We then discuss one possible solution (out of many) under specified assumptions.

11. NOTATION

Suppose there exist n initial incumbent machines. In order to define the model, the following

the utilization rate of the j t h machine in the sequence of replacements for the ith initial incumbent machine, i = 1, . . . , n , j = 1, 2, . . . (from here on the underlined is abbreviated by MRIMij) ; the age of MRIMij when the decision to replace it is made; the age of MRIM, when it is replaced; the age of MRIMij, 0 c 7 c l i e .I’

the utility class of MRIMij (Cj=1, 2, . . ., q in decreasing order of desirability. That is, each machine is represented by an ordinal measure of its utility characterized by one of the above integers. Moreover, every machine belongs to one and only one utility class at any point in time.); a parameter indicating whether MRIMij is new or a modernization

notation is useful:

u:,

Lj, J’

7,

ci J’

Rj ,

*This work was started while the author was on the professional staff of the Center for Naval while the author was a visiting professor at the Naval Postgraduate School.

547

Analysis. It was completed

548 P. J. KALMAN

0 if MRIMij is a modernization 1 if MRIMij is new (Rj={

the time when a decision to purchase a replacement for MRIMij is made, 0 c t i . I s 00;

xi ( t ) 9

X h ( t ) 3

Pkj (7) 7

the expected number of machines in the ith sequence in utility class k at time t; the expected number of machines in utility class k (k= 1, . . ., q ) at time t (0 t w ) ;

the probability that a machine which was in class k when it began operating will have moved to c lass j by the time it is T years old; the time the latest replacement machine in the ith sequence began operating; the starting time of the latest modernization in the ith sequence.

In the model to be presented, the term “replacement” includes “modernization” as well as new machine acquisition. The same applies to “purchase of a replacement”. Hence, if MRIM, is “modern- ized”, then the modernized version is called “new” and a “new machine” when it enters the process. It follows that “age” represents “duration of time in the process”. The symbols Lj, CJ and R; represent decision variables and Cl refers to utility class when new.

t ’ , t l , I

111. THE MODEL The model defined below allows a three-way choice among not replacing or modernizing or building

a new machine at any point in time. Furthermore, the model will determine a sequence of replacement purchase times (i.e., { t j } , j = 1, 2, . . ., for each i, i = 1, . . ., n) such that the total “current account” cost of all future costs over an infinite time horizon is minimized subject to the constraints that at each point in time there exist a prescribed expected number of machines in each chosen utility class k ( k = 1, 2, . . ., q ) . It is assumed that there exist n initial incumbent machines.

The operating expence (cost) function for MRZMij is

where it is understood that the superscript outside a function applies to all the appropriate arguments of that function. Clearly, uj, T, Rf and Cf influence the operating expense of MRIMij. The time at which the decision to replace is made (tj-,) determines the technological state of advancement of the machine to be installed and hence can also influence + i .

In addition to operating cost, there is one other major category of cost which must be considered- the investment cost (Wj) of MRIMij. This will depend upon whether it is a new machine or a modern- ization and on the utility class chosen for it. That is,

(3.2) Wj(Rj, Cj).

Since machines normally have some salvage value at the time of replacement, Wj is not the net capital cost of a machine. To obtain this, we subtract the salvage value (which depends on the replacement age) of MRIMij. Hence, net capital cost of MRIMij is

(3.3)

We like to note that salvage value may also depend on absolute time, but for simplicity we omit this. There is one other important type of element to be considered before one can formulate the “current

549 STOCHASTIC CONSTRAINED REPLACEMENT MODEL

account” cost. This is the time required to accomplish a replacement of MRIMij of age L>if the replace- ment action starts at time tf. Before defining this relation, we like to point out that if a machine is to be replaced by an acquisition, it will remain in service till the new arrival, whereas it may be immedi- ately withdrawn if it is to receive modernization. The time required to accomplish a replacement of MRIMij of age L$ if the replacement action starts at time t$ is represented by

The L j and the tj are related by

t;=Li 0’

t i 1 = t i 0 +Li 1 + Y:(L0, to)

. . tZ=t* J j - 1 +y! J - 1 ( , - 1 , t j - l )+ f , j 3 2 , i = l , 2 , . . . ,n .

Yji(Lj, t j ) , j = o , 1 , 2 , . . .) i = l , . . .) n.

For future notational convenience, let -yj represent

Define r as the continuous rate of discount. Also, define the current account cost (wi) as the outlay stream that has the same present value as all the cost outlays associated with the ith initial incumbent machine in an infinite chain of replacements. That is,

where i = l , . . ., n,

lh+l= Lh+l if the (m+ 1)st machine’s replacement in the ith sequence is a modernization,

Lh+l + yh+l if a new machine is built, m=- 1, 0 , 1, 2, . . ., and c#$(u,, T, R, , C , ) is the oper- ( ating expense function for the ith incumbent machine.

Note that the investment cost of the ith incumbent machine, WL, does not appear in (3.4) since, by

hypothesis, the incumbent machine is a sunk investment whose capital cost should not be allowed to influence decisions. The integrals in (3.4) discount the cost stream of each replacement back to the point in time when the machine was new. The exponential term outside each set of brackets then dis- counts all these cost streams back to the present time. Similarly, the second and third terms inside

P. J. KALMAN 550

each bracket, when discounted, determines the present value of all future investment outlays net of salvage values. Finally, I would like to note that equation (3.4) can be reduced to its discrete analogue in the dynamic programming framework under appropriate assumptions.2

The constraint set will now be formulated. It will be formulated first as an algebraic system and then as a differential equation system. The choice of which formulation to use will depend upon the application of the model. From the definitions (of section 2) it follows that

(3.5)

j = 1 , 2 , . . .) 4, with the constraint

where ;Cj is chosen by the decision maker (we specify a way for calculating the x’s below), and

A . - . [ t i ] = max { t l , t l } ,

1 if the machine in use at time t in the ith incumbent machine’s sequence is in utility c lass j if it is not.

Note that t J ( t ) is a function of current time t. If a machine in the ith sequence is in the process of being modernized at time t , then t j ( t ) = O for allj. Consider the inner summation first. tJ( [ t i ] ) indicates the utility class which we selected for the machine currently in use at time t in the ith sequence when it was new, or tells us that a modernization is currently being undertaken, and consequently, that no machine in the ith sequence is currently in use (i.e., t i ( [ t i ] ) = 0 for all k). For sequence i in which modernizations are not being undertaken at time t , t i ( [ t i ] ) =t i ( 2 ‘ ) = 1 for one and only one k. If [ t i ] = i i , then t - [ t i ] gives the age of the machine in use at time t in the ith sequence. If [ t i ] = t : the p k j ( t - [ t i ] ) term is irrelevant since t i ( [ t i ] ) = 0 for all k. Thus, the inner sum tells us, for each sequence i in which a machine is in use at time t , the probability of that machine being in utility c lass j at time t , given the class of the machine when it was built. Then, with the outer summation, we simply sum over all machines in use at time t.

In the above formulation, the constraints are functions of the replacement purchase times ( [ t ’ ] is either a replacement purchase time or a time at which a replacement machine begins operation, which is related to a replacement purchase time by the function y i ) and the utility classes which we choose for the replacement machines, as reflected in the values of eL( ). The constraints may have a count- able number of discontinuities in an infinite horizon model, since it is likely that there will be a ‘bump’ every time we are able to choose a utility class for a machine (i.e., when we have a modernization or build a new machine).

An alternative formulation of the constraint set will be given via a system of differential equations. This formulation involves the solution to a differential equation system while the above formulation does not.

- .

For each sequence i, ( i = 1, . . ., n) we have the following matrix differential equation:

(3.7) - P ( t - [ t i ] ) X i dx‘ dt --

55 1 STOCHASTIC CONSTRAINED REPLACEMENT MODEL

where P is a q x q transition probability matrix of transition probabilities P i j ( t - [tf]), X i is a q x 1 matrix of x)(t)'s, subject to the initial condition

(3.8) Xi( [ti]) = X i ( O )

where Xi is a q x 1 matrix whose j th element is x)( [ti]) where

1 if [ t i ] = i i and we selected utility class j

for this machine when it was built,

0 otherwise.

That is, if modernization is going on at time t in the ith sequence, we do not have to be concerned about any transition probability between classes because no machine is in use. Otherwise, we look back to see in what class we put the machine in use at time t in the ith sequence when it was new. In system (3.7) the j th equation

states that the net rate of change of the number of machines in ith sequence in classj in an infinitesimal time period equals the total number of machines which enter classj in the i th sequence at time t minus the number of machines in the ith sequence which leave c lass j for other classes.

If we assume that the matrix P is an analytic coefficient matrix, then our finite system of first-order linear differential equations with analytic coefficient functions (3.7) subject to the initial value (3.8) admits a unique analytic-function solution, which can be found explicitly by equating coefficients in the relevant power ~ e r i e s . ~ Clearly, there are an infinite number of functions in this class which could be used to represent the transition probabilities as a function of time. Linear or exponential functions are just two elements of this class which we can use.

One should note that we have a set of q differential equations for each sequence i. Let us denote:

TNDMi, the time of the next decision to modernize a machine in the ith sequence; TNROi, the time the next replacement machine begins operation in the ith sequence.

For fixed i, we may have a different set of initial conditions for each interval of the form [ti] 6 t < min { TNDMi, TNROi}, reflecting the series of decisions we can make concerning the class of the replacement machines. Thus, for each i, we would have at most q non-trivial sets of differential equa- tions to solve, corresponding to the q different non-trivial initial conditions we may have. Of course, a solution corresponding to a given set of initial conditions may apply to different length intervals of the form { t i } s t < min {TNDMi, TNROi}.

Thus, the xf(t) will be functions of the replacement purchase times and the utility class decisions.

t

See [Z], pp. 89-92.

552 P. J . KALMAN

Then x j ( t ) = n

i= 1

x ) ( t ) . However, we could only hope for continuity of x j ( t ) for

max [ t i ] s t < min {TNDMi, TNROi}.

That is to say, x j ( t ) would be continuous only in intervals tE n ( [ t i ] , min {TNDMi, TNROi}). The

same is true of the continuity of x j ( t ) in our non-differential equation approach. For example, suppose we have two machines

i i

i i

Machine 1

Machine 2

Specifically, suppose at tz a decision is made to build a new machine in sequence 1, while < involves a decision to modernize the machine in use in sequence 2 . Thus [ t ’ ] = t l , [ t 2 ] = t ; Thus

n ( [ t i ] , min {TNDMi, TNROi}) = ( t ; , t3 ) , i i

as denoted by the dotted lines. It is within this interval that no decisions regarding the utility class of the machines are made. Consequently, no “bumps” are experienced. Instead, machines change classes smoothly according to the transition probabilities.

It should be emphasized that the main factor behind the two different approaches to the constraint formulation is the definition of the transition probabilities. It seems likely that the first approach is preferable from a practical standpoint.

If the transition probability matrix is a constant matrix P it is well known that system (3.7) has the following unique solution

I . m i (3.12)

where P I , . . . , Pk are the eigenvalues of P with multiplicities m l , . . . , mk, respectively and vlj is the eigenvector associated with pj. The solution (3.12) represents the number of machines in class 1 ( l = l , . . . , q ) in thei th sequence at timet.

We now formulate the constrained optimization problem of determining a sequence of replacement times such that the total current account cost of all future costs and capital expenditures over an infinite time horizon of the n incumbent machines is minimized subject to the chosen constraints that & ( t ) machines are in utility worth class k at time t ( k = 1, . . . , q ) . That is, we want to minimize

5 wi where w z is defined by (3.4) subject to the constraints x j ( t ) = T j ( t ) , j = 1, . . . , q which are

defined by (3.5). One possible way to proceed is to assume that the constraints are exactly satisfied for all values of t and make a discrete approximation of the objective function. Then we can apply dynamic programming techniques in order to solve the problem.

i = l

As an exercise we have solved the model under the following assumptions:

(1) a constant level of utility over time; (2) a linear operating cost function;

STOCHASTIC CONSTRAINED REPLACEMENT MODEL 553

(3) a constant rate of machine utilization; (4) instantaneous replacement; (5) equal economic lives of all replacements after the first; (6) equal investment costs of all machines.

IV. ACKNOWLEDGMENT I would like to thank G. E. Bowden, L. Ravenscroft and D. Richter, who are members of the

professional staff of the Center for Naval Analysis for their helpful comments.

REFERENCES

[l] Dreyfus, S., “Dynamic Programming and the Calculus of Variations,” Academic Publishing Co.,

[2] Hochstadt, Harry, “Differential Equations: A Modern Approach,” Holt, Rinehart, Winston Pub-

[3] Hotelling, H., “A General Mathematical Theory of Depreciation,” Journal of the American Statistical

[4] Jorgenson, D. W., J. J . McCall, R. Radner, “Optimal Replacement Policy,” Rand McNally Pub-

1965.

lishing Co., 1964.

Association, p. 340, XX, September 1925.

lishing Co., 1967.