Heterogeneous Multiserver Queues with General

Input

Winfried K. Grassmann

Department of Computer Science

University of Saskatchewan

57 Campus Drive

Saskatoon, SK S7N 5A9

Yiqiang Q. Zhao

Department of Mathematics and Statistics

University of Winnipeg

515 Potage Ave

Winnipeg, MB R3B 2E9

September 2, 2004

Abstract

A queueing system with heterogeneous servers and general input is analyzed. In par-

ticular, it is shown how to obtain the steady state probabilities for such a system, both at

random-times and at the times preceding an arrival. If more than one server is idle, an ar-

rival has the choice how to select one of the idle servers. The rules used to do the selection

is part of the model description. The probabilities at arrival points are found as follows.

First, one determines the probabilities for all states in which there is no queue, and then

one calculates the probabilities in which customers are waiting. The probabilities at arrival

epochs allow one to find the probability of no wait, the waiting time distribution, and the

distribution of the number in the system at random epochs. Heuristic arguments, sup-

ported by numerical calculations, show that the importance of selecting a server increases

with decreasing traffic intensity, increasing number of servers, and decreasing variation

between interarrival times.

Keywords: Queueing, heterogeneous servers, general input, GI/M/c

1

1 Introduction

The system investigated in this paper contains c servers, labelled s1, s2, . . . , sc. All service

times are exponential, and the service rate of server si is µi, i = 1, 2, . . . , c. The times

between arrivals are independent and identically distributed random variables with a common

distribution A(t). In Kendall notation [10], we therefore deal with a GI/M/c queueing system,

except that the servers are heterogeneous. Our objective is to find the steady state distribution

of the number of elements in the system. We assume that such a distribution exists, which is

the case if and only if µ > λ, where µ = µ1 + µ2 + · · ·µc, and λ is the arrival rate.

The investigation of queueing systems with heterogeneous servers is called for because in

many queueing systems arising in practice, the individual servers work at different speeds. All

humans are different, and if the servers are human, one can expect that they work at different

rates. But even if the servers are machines, as in production plants, it is very well possible

that some machines are faster than others. The plant may have cheaper, or possibly older

machines which are used when traffic is so heavy that it would otherwise exceed the capacity.

In communications systems, in fact, advances in technology are so rapid that older equipment

is often significantly slower than newer one.

In literature, one distinguishes several server selection rules. Gumbel [11] deals with ran-

dom selection in which arrivals choose any free server with equal probability. If all servers are

busy, arrivals wait. Gumbel also assumed Poisson arrivals. Other authors [4, 7, 13, 15, 17, 22]

discuss the ordered entry discipline, in which arrivals choose the first free server. This disci-

pline arises in the context of conveyor belts. Most of these papers deal with loss systems. An

exception is a paper by Nawijn [15]. Nawijn assumes homogeneous service times, however.

Other authors use general selection rules, as does this paper. In particular, Yu [25] assumes

that if there are free servers, arrivals select one of the free servers with a certain probability, a

probability which depends on the present state of the system. If all servers are busy, arrivals

join a queue. A number of researchers have investigated optimal rules for server selection

[18, 5, 21, 19]. Of course, the homogeneous case of the GI/M/s queue has been analyzed

repeatedly (see [8] and references), in particular by DeSmit [6] and Takács [23].

For analysing the GI/M/c queue with heterogeneous servers, a good notation is essential.

Both Yu [25] and Neuts [16] based their notation on Kronecker products. We use set theory

1

instead, a notation which compact, yet powerful. We have to admit, though, that Yu solves

the GI/Ek/c queue, whereas Neuts solves the GI/PH/c queue, systems more complex than

the GI/M/c queue. However, we believe that many of our methods can be generalized to the

GI/PH/c queue by using bags rather than sets, but this is beyond the scope of this paper.

Both Yu and Neuts use an imbedded Markov chain approach, and they derive the tran-

sition probabilities of this chain by integration. Here, we avoid integration by using Laplace

transforms, an approach which is traditionally used in the case of a single server, but which

also can be used for multiple servers [23]. To find the random-time probabilities, Neuts uses

integration once more. Yu does not derive any random-time probabilities. We calculate the

random-time probabilities by using transforms. For some performance measures, we were able

to simplify the calculations of random-time performance measures by using Little’s formula.

The calculations to obtain arrival-time and random-time probabilities are non-trivial, as

evidenced by a paper by Avis [1]. Avis uses trees to represent the state of the system, and he

provides a number of theorems to justify his algorithms. In the place of trees, we use binary

numbers for dealing with the states of the system. All the operations of interest can then be

expressed by bit manipulations. In this way, the complexity of the program can be reduced

considerably.

The methods given by Yu and Neuts require a state space that increases exponentially

with the number of servers, and so does the method given in this paper. All these methods

require one to solve 2c equations in 2c variables, where c is the number of servers. This means

that c cannot be large. Our calculation were done for up to 10 servers, and even with very

fast computers, our methods do not allow one to go much beyond that.

The outline of the paper is as follows. Section 2 contains the notation used throughout

the paper. Section 3 shows how to calculate the pertinent probabilities immediately before an

arrival. These probabilities allow one to find the random-time probabilities as shown in Sec-

tion 4. Section 5 derives a number of important measures, both for random and arrival-times.

These relations form the basis of Section 6, in which we try to gain a qualitative understanding

of the system through heuristic arguments and numerical experiments. Section 7 highlights

some computational issues.

It is unavoidable to use advanced mathematics in a paper dealing with the GI/M/c queue.

In particular, Sections 3 and 4 may prove challenging. These sections provide the basis of

2

our programs, and hence the basis of our experiments, and they are in this sense crucial. To

accommodate readers not intending to write algorithms or to do research in the area, we have

written the paper such that these sections can essentially be skipped. For readers desiring to

do numerical experiments on their own, the senior author has written a program in C that is

available from him.

2 Notation

Our first task is to define the state space of the system in question. The description of the

states depends on whether or not there is a queue. If there is a queue, that is, if there are

some elements waiting to be served, then the state is given by the queue length, that is, by

the numbers 1, 2, 3, . . . . If there is no queue, the state is given by the set of busy servers.

Specifically, if C = {s1, s2, . . . , sc} is the set of all servers, then all subsets of C must be

considered as states. Hence, while there is no queue, the set of states is given by 2C , the

power set of C. It is convenient to consider state C to be identical to state 0, because in state

C, all servers are busy, while the queue length is 0. Hence, E = 2C is the set of states with

no queue, and F = {0, 1, 2, . . .} is the set of states in which all servers are busy. The state

space U is E ∪ F . The state 0 = C belongs to both E and F .

The algebraic operations defined for natural numbers are different from the algebraic

operations involving sets, and this creates problems here, because operations on E involve

sets, and operations on F involve natural numbers. For instance, if i, j ∈ E, then i ⊆ j is

meaningful, but if i, j ∈ F , it is not. In the case where i, j ∈ F , the ⊆ relation must always

be replaced by ≤. To bypass this difficulty, we will interpret ⊆ from now on as ≤ if i, j ∈ F .

Moreover, any state in E is considered to be a subset of any state in F . We also define i ⊂1 j

as follows. If i, j ∈ E, i ⊂1 j means that i ⊂ j and i contains exactly one element less than in

j, and if i, j ∈ F , then i ⊂1 j means that i = j − 1. For each state G ⊆ C, we define

µG =∑

sk∈G

µk . (1)

Clearly, µC = µ.

The interarrival time distribution is given as A(t). In the formulas to be derived, we will

3

need the Laplace-Stieltjes transform of A(t):

L(s) =

∫ ∞

0e−stdA(t) . (2)

The arrival rate λ is, of course, −1/L′(0). As usual, ρ = λ/µ. If I ⊂ J ⊆ C, qIJ indicates

the probability that in state I, an arrival selects one of the servers, which results in state J .

If the queue is an ordered entry queue, and if in state I, sk is the first busy server, then J is

I ∪ {sk}. For the case where service is in random order, let |I| be the cardinality of I. qIJ is

then given by

qI I∪{sk} = 1/(c − |I|) sk 6∈ I

qIJ = 0 otherwise.

Other selection rules can be expressed in a similar way. Note that in all cases, I ⊂1 J because

the arrival chooses only one server. If all servers are busy, then qij = 1 if j = i + 1, and zero

otherwise.

In the embedded Markov chain approach used here, we consider the state immediately

before the nth arrival. This state will be denoted by Xn. {Xn; n > 0} forms a Markov chain,

and the steady state probabilities of this chain are denoted by πj , that is

πj = limn→∞

P (Xn = j), j ∈ U .

All πj can be found from the equilibrium equations, which are given by

πj =∑

i∈U

πipij , j ∈ U . (3)

Here, the pij are the transition probabilities, that is

pij = P{Xn+1 = j|Xn = i} i, j ∈ U . (4)

How to find the pij is shown in the next section. Section 4 then derives the random-time

probabilities Pj , j ∈ U , from the arrival-time probabilities πi. For every state j ∈ U , Pj

represents the probability that an outside observer sees the system in state j, given he/she

inspects the system at a moment chosen at random. Alternatively, Pj can be interpreted as

the proportion of time the system spends in state j.

4

3 Arrival Time Probabilities

To evaluate the transition probabilities pij as defined in (4), consider the system at three

epochs: the epoch immediately before the nth arrival, the epoch immediately after the nth

arrival, and the epoch immediately before the n + 1st arrival. The states of the system at

these epochs are denoted by Xn, X+n and Xn+1. The arrival converts state Xn to state X

+n ,

and from X+n , the system makes a transition to Xn+1. By definition

P{X+n = k|Xn = i} = qik, i, k ∈ U .

Define

rkj = P{Xn+1 = j|X+n = k}, i, k ∈ U.

The transition probabilities pij , i, j ∈ U can then be found as

pij =∑

k∈U

P{Xn+1 = j, X+n = k|Xn = i}

=∑

k∈U

P{Xn+1 = j|X+n = k}P{X

+n = k|Xn = i}

=∑

k∈U

qikrkj . (5)

To determine rij , let Yn(t) be the state of the system t time units after the nth arrival, given

all arrivals after the nth arrival are ignored, and define for all i, j ∈ U

rij(t) = P{Yn(t) = j|X+n = i} .

Given rij(t), one finds rij as

rij =

∫ ∞

0rij(t)dA(t) . (6)

Three cases must be distinguished when determining the rij(t), depending whether (1) i, j ∈ E,

(2) i, j ∈ F and (3) i ∈ F , j ∈ E.

If I, J ⊆ C, then both I and J are in E, and rIJ(t) is the probability that the servers

in set J are still busy t time units after the system was in state I, while the servers of the

remaining set I − J have become idle. Since the probability that server sk, sk ∈ I has not

completed service at time t is e−µkt, one has

rIJ(t) =∏

sk∈J

e−µkt∏

sm∈I−J

(1 − e−µmt) , J ⊆ I ⊆ C. (7)

5

If i, j ∈ F , then the system is full, and while the system is full, the number of customers

served in a time interval of length t follows a Poisson distribution with expectation µt. Hence

P{n customers served} = p(n; µt) = e−µt(µt)n/n!, n ≥ 0 .

Consequently

rij(t) = P{i − j served in t} = p(i − j; µt) = e−µt(µt)(i−j)/(i − j)! , i, j ∈ F . (8)

If i ∈ F , and J ⊂ C, we first have to reach state C before entering state J . State C is reached

between τ and τ + dτ if the system is in state 1 at time τ , and makes a transition from 1 to

C in the interval dτ . By multiplying the probabilities of these events, one finds

P{τ ≤ Time to reach C < τ + dτ} = µp(i − 1; µτ)dτ .

Hence

riJ(t) =

∫ t

0µp(i − 1; µτ)rCJ(t − τ)dτ , i ∈ F, J ⊂ C . (9)

Equations (7), (8) and (9) determine the rij(t) for all possible cases. The rij can then be

obtained from (6). The left-hand side of (6) can be integrated by numerical methods (see e.g.

[16], where integration is applied to solve similar problems). However, numerical integration

has certain disadvantages. The precision of the results may suffer because of discretization

errors. In addition to this, there is a truncation error because the improper integrals in

question must be approximated by proper integrals. For this reason, we suggest the following

method. Write (7) as

rIJ(t) =∑

G⊆I−J

(−1)|G| e−tµJ∪G

, J ⊆ I ⊆ C . (10)

Here, |G| is the number of elements in G, and µG is given by (1). In view of (6) and (2), (10)

yields

rIJ =

∫ ∞

0rIJ(t)dA(t) =

∑

G⊆I−J

(−1)|G|L(µJ∪G), J ⊆ I ⊆ C . (11)

When the system is full, its transition matrix is identical to the one for the GI/M/1 queue,

and, as in the GI/M/1 queue, one has [10]

πj = πCξj , (12)

6

where ξ is found from the following equation

ξ = L((1 − ξ)µ) . (13)

We will also need the following equation which can be derived readily

∞∑

i=0

ξip(i; µt) = e−µt(1−ξ) . (14)

Because of (12), we only need to solve the boundary equations, which become, in view of

(3), (5), and (12)

πJ =∑

I⊂C

πI∑

I⊂1K

qIKrKJ + πC

∞∑

i=0

ξiri+1 J . (15)

We use (6), (9), (10) and (14) to obtain

∞∑

i=0

ξiri+1 J =∞

∑

i=0

ξi∫ ∞

0

∫ t

0µp(i; µτ)rCJ(t − τ)dτ dA(t)

=

∫ ∞

0

∫ t

0µe−µτ(1−ξ)rCJ(t − τ)dτ dA(t)

=

∫ ∞

0

∫ t

0µe−µτ(1−ξ)

∑

G⊆C−J

(−1)|G|e−(t−τ)µG∪J

dτ dA(t)

= µ∑

G⊆C−J

(−1)|G| LG∪J . (16)

Here, LK is given as

LK =

∫ ∞

0

∫ t

0e−µτ(1−ξ)e−(t−τ)µ

K

dτ dA(t)

=

∫ ∞

0e−tµ

K

∫ t

0e−τ((1−ξ)µ−µ

K)dτ dA(t) . (17)

Straightforward integration of the inner integral, together with (2) and (13) yields

LK =L(µK) − ξ

(1 − ξ)µ − µK.

If µK = (1 − ξ)µ, one uses L’Hôpital’s rule, with µK as the variable, to obtain

LK = −L′(µK) .

To avoid problems of numerical instability, it is also recommended to use this formula when

the difference between µK and (1 − ξ)µ is small compared to µ.

7

The equilibrium equations for J ∈ E become now

πJ =∑

I⊂C

πI∑

I⊂1K

qIKrKJ + πCµ∑

G⊆C−J

(−1)|G|LG∪J . (18)

These equations determine πJ for all J ⊆ C. The πj , j ≥ 0 can then be found from (12).

4 Random Time Probabilities

We first review the general method to obtain random-time probability as given by Gross and

Harris [10, page 349-351], and their formulas will then allow us to derive the random-time

probabilities from the arrival-time probabilities. First, mj is defined to be the expected time

between renewals, and the following expression is formed:

vj =πjmj

∑

i∈U πimi.

For the probabilistic interpretation of vj , see [10]. In our case, mi = 1/λ for all i. Therefore,

vi = πi. One now finds the probabilities Pj , the probability of being in state j at a random

point in time as follows

Pj =∑

i∈U

vi

∫ ∞

0uij(t)A

c(t)/midt . (19)

Here, uij(t) is the probability of being in state j t time units after the renewal, given the state

immediately preceding the renewal is i, and Ac(t) is the probability that the time between

renewals exceeds t. In our case, the times between renewals are the times between arrivals,

that is, Ac(t) = 1 − A(t). One can instantiate uij(t), i ⊇ j as follows:

uij(t) =∑

i⊂1k

P{X+n = k, Yn(t) = j|Xn = i} =∑

i⊂1k

qikrkj(t) .

For i ⊂ j, uij(t) = 0. Hence, Pj as given in (19) becomes

Pj =

∫ ∞

0

∑

j⊆i

∑

i⊂1k

πiqikrkj(t)λAc(t)dt .

For j ≥ 0, the derivation is as in the GI/M/1 queue [10]:

Pj =λ

µξj−1πC =

ρ

ξπj . (20)

8

The case J ⊆ C is more challenging. Specialising (19) yields

PJ = λ

∑

I:J⊆I⊆C

∑

I⊂1K

πIqIK

∫ ∞

0rKJ(t)A

c(t)dt +

∫ ∞

0

∞∑

i=0

πCξiri+1 J(t)A

c(t)dt

. (21)

To bring∫

rKJ(t)Ac(t) into a convenient form define

Φ(s) =

∫ ∞

0e−stAc(t)dt . (22)

One has, as is well known

Φ(s) =

1s (1 − L(s)) , s 6= 0

1λ s = 0 .

(23)

In view of (10) and (22), one obtains

∫ ∞

0rKJ(t)A

c(t)dt =

∫ ∞

0

∑

G⊆K−J

(−1)|G| e−tµG∪J

Ac(t)dt

=∑

G⊆K−J

(−1)|G|Φ(µG∪J) . (24)

This provides a solution of the first integral of (21). For the second integral, proceed as in

(16) and (17) to obtain

πC

∫ ∞

0

∞∑

i=0

ξiri+1J(t)Ac(t)dt = πCµ

∑

G⊆C−J

(−1)|G|FG∪J . (25)

Here, FK is defined as

FK =

∫ ∞

0e−tµ

K

∫ t

0e−τ [(1−ξ)µ−µ

K ]dτAc(t)dt .

Solving the inner integral yields

FK =1

µ(1 − ξ) − µK

∫ ∞

0(1 − e−t[µ(1−ξ)−µ

K ])e−tµK

Ac(t)dt

=1

µ(1 − ξ) − µK(

Φ(µK) − Φ(µ(1 − ξ)))

.

Now, because of (13) and (23), one has

Φ(µ(1 − ξ)) =1

µ(1 − ξ)(1 − L(µ(1 − ξ)) =

1

µ.

9

One concludes

FK =1

µ(1 − ξ) − µK

(

Φ(µK) −1

µ

)

. (26)

For the case that µK = µ(1− ξ), one uses L’Hôpital’s rule. After some calculation, using also

µK = µ(1 − ξ), one obtains

FK =1

µK

(

L′(µK) +1

µ

)

. (27)

Hence, for all J ⊆ C:

PJ = λ∑

I:J⊆I⊆C

∑

I⊂1K

πIqIK∑

G⊆K−J

(−1)|G|Φ(µG∪J) + πCλµ∑

G⊆C−J

(−1)|G|FG∪J . (28)

Here, Φ(µK) is given by (23), and FK is given by (26) or (27), respectively. Equation (28)

determines all PJ , J ⊆ C. The Pj , j ≥ 0 are given by (20).

5 Performance Measures

Given the arrival-time probabilities πi and the random-time probabilities Pi, i ∈ U , a number

of interesting statistics can be calculated. Before doing this, note the following relations

between the πi and the Pi. If all servers are busy, the random-time probabilities can easily be

obtained from (20), with ξ determined from (13). For states with some idle servers, the relation

between arrival and random-time probabilities is given by (28), a rather complex formula.

Some simple formulas for calculating certain random-time measures will be introduced in this

section. They are based on Little’s formula.

There are a number of measures that are relevant to the customer. These measures can

be obtained from the πi, i ∈ U . A customer has to wait if at her arrival, all servers are busy.

Hence, if B is the probability a customer has to wait, one finds, since πi = π0ξi

B =

∞∑

i=0

πi =

∞∑

i=0

π0ξi =

π0(1 − ξ)

. (29)

Because of (20), the probability Br that all servers are busy at random time is

Br = Bρ/ξ . (30)

Consider now Wq, the expected time in queue. If there are i elements in the queue at the

moment of an arrival, there must be i+1 service completions: all customers presently waiting

10

must be served, and one server must become free. Since (i + 1)/µ time units are needed to

serve i + 1 customers, this yields

Wq =∞

∑

i=0

(i + 1)πi/µ =∞

∑

i=0

(i + 1)π0ξi/µ = π0(1 − ξ)

−2/µ . (31)

In view of (29), this can be written as

Wq =B

(1 − ξ)µ. (32)

Lq, the expected number in queue at random-time, can now be obtained by Little’s formula:

Lq = Wqλ =Bρ

1 − ξ. (33)

The same result can be obtained by using (20) and (12).

The probability that the time in queue, Tq, exceeds t is given as

P{Tq > t} = Be−µt(1−ξ) . (34)

This can be proven using similar techniques as are used in the M/M/1 queue [10, page 76].

Incidentally, (34) gives a new interpretation to L(µ(1 − ξ)). According to (2) and (13),

one has

ξ = L(µ(1 − ξ)) =

∫ ∞

0e−tµ(1−ξ)dA(t) . (35)

Inside the integral, one recognises the waiting time distribution of an arrival, given the arrival

has to wait. Consequently, L(µ(1 − ξ)), or, equivalently, ξ is the probability that given an

arrival has to wait, the next arrival comes before the waiting time is over.

Next we will calculate E(D), the average time an arbitrary arrival spends in service. To

calculate the E(D), two cases must be distinguished. In the first case, the arrival occurs while

a server is idle, The expected service time is 1/µk if the arrival selects server sk, and this

happens with probability qI I∪{sk}. Hence

E(D|I) =∑

sk 6∈I

qI I∪{sk}/µk .

In the second case, the arrival finds all servers busy. There is a probability µk/µ that the

arrival will eventually be served by server sk, and the expected service time of server sk is

1/µk. It follows

E(D|F ) =c

∑

k=1

(µk/µ)(1/µk) = c/µ .

11

By the law of total probability, one therefore has

E(D) =∑

I⊂C

πI∑

sk 6∈I

qI I∪{sk}/µk + Bc/µ . (36)

Equations (29), (32) and (36) allow one to express the sojourn time W as

W =B

µ

(

1

1 − ξ+ c

)

+∑

I⊂C

πI∑

sk 6∈I

qI I∪{sk}/µk .

As ρ goes to zero, both B and ξ go to zero, which means that the second term becomes

dominant. Similarly, as ρ goes to one, B and ξ both converge to one, and the first term

becomes dominant.

It is also possible to find the distribution of the time of a customer is spending in the

system. The difficulty here is that the waiting time and the service time are not independent.

Specifically, customers arriving when the system is idle have different service time distributions

than customers arriving when all servers are busy. Because of this, one must calculate the

waiting time distribution by conditioning on the state of the system before the arrival. If the

system is in state I ⊂ C, then the time a customer is spending in the system is a mixture of

exponentials. Otherwise, it is the convolution of the waiting time with the service time. If TS

is the time a customer is spending in the system, this yields:

P{TS > t} =∑

I⊂C

∑

sk /∈I

πIqI I∪{sk}e−µkt + BP{TS > t |busy} ,

where,

P{TS > t |busy} = P{TS > t , Tq > t |busy} + P{TS > t , Tq ≤ t |busy}

= e−µt(1−ξ) +c

∑

k=1

µkµ

P{TS > t , Tq ≤ t |busy, served by sk }

with

P{TS > t , Tq ≤ t |busy, served by sk } =

∫ t

0e−µk(t−x)µ(1 − ξ)e−µ(1−ξ)xdx .

This expression can be integrated without difficulty, and the problem is solved. When inte-

grating, one must treat the case µk = µ(1 − ξ) separately.

It is possible to use Little’s formula to find the random time expectation for the number

of customers receiving service as follows

E(Customers in Service) = E(D)λ .

12

Little’s formula can also be applied to the individual server, where it leads to the following

relation

P{sk busy} = λk/µk . (37)

Here, λk is the expected number of customers handled by server sk per time unit. To prove

(37), note that the expected time a customer spends with server sk is 1/µk, and the expected

number of customers with sk is therefore λk/µk. Since the average number of customers served

by an individual server sk is either zero or one, this average is equal the probability that sk

is busy, and (37) follows. λk, in turn, is λ times the probability that a customer is served by

sk. This probability is given as:

P{sk serves arrival} = Bµk/µ +∑

sk 6∈I

πIqI I∪{sk} . (38)

To prove (38), note that customers arriving while the server is busy have a probability of µk/µ

to be served by sk. Those arriving while at least one server is idle choose sk with probability

qI I∪{k}. We now use (37) and (38) to find the probability that server sk is busy. Alternatively,

one can find this probability by using the PI and (30).

P{ sk busy } = ρ(

B +∑

I⊂C

πIqI I∪{sk}µ/µk)

= Bρ/ξ +∑

sk∈I, I⊂C

PI . (39)

6 The Behaviour of the System

The system behaves quite differently depending on whether there are idle servers or not. For

discussing the system while it is busy, it is sufficient to know the parameters µ and ρ, and the

probabilities ξ and πC . This is implied by formulas (20) and (29)-(34). In particular, there

is no need to know the service rates of the individual servers, as long as their sum is µ, and

there is no need to know the way arrivals select servers, provided πC is known. The values of

the µk, the qIK , the πI and PI , I ⊂ C are only needed when there are idle servers.

There is some literature regarding ξ. Şahin [20] provided tables for ξ for Erlang and

deterministic arrivals. Al-Khayyal and Gross [2] related ξ to the coefficient of variation of the

time between arrivals, which they denoted by Cv. They found that ξ increases with both ρ

and Cv. Moreover, they found that if Cv < 1, then ξ tends to be less than ρ, and if Cv > 1,

then ξ > ρ. As ρ approaches 1, ξ approaches ρ.

13

We think that the crucial point here is not the value of ξ itself, but h = ξ/ρ. The reason

for this is that according to (20), πj = Pjh for j ≥ 0, that is, h determines how the arrival-

time probabilities in the busy system compare to the random-time probabilities. This leads

to the question why the arrival probabilities may differ from the random-time probabilities.

According to (35), ξ is the probability that given the nth arrival must wait, the n+1st customer

arrives while the nth customer is still waiting. A little thought indicates that if arrivals are

unevenly spaced, then there is a great likelihood that a customer arrives during the waiting

time of the previous arrival, and that she therefore has to wait. The probability that this

happens is presumably reduced if arrivals are spaced evenly. This suggests that if arrivals

are deterministic, they should encounter less congestion than Poisson arrivals, which in turn

should encounter less congestion than hyper-exponential arrivals. This agrees with the findings

of Gross and Khayyal. Table 1 gives quantitative evidence to this fact. The first column in

this table gives h, which is also the ratio of πi/Pi for all states with c or more elements in the

system. From this table, it is clear that for low ρ, h can be significantly different from 1, that

is, the arrival-time probabilities can be significantly different from random-time probabilities.

Even for ρ as high as 0.5, the arrival-time probabilities in the case of deterministic arrivals

are almost 60% below the random probabilities.

The hyper-exponential distribution is given by [2]

a(t) = 2p2λe−pλt + 2(1 − p)2λe−2(1−p)λt .

Here, t > 0 and 0 < p ≤ 0.5. The coefficient of variation is

Cv =

√

1

2p(1 − p)− 1 .

In Table 1, we use the hyper-exponential distribution with p = 0.1. For this value of p, and

for ρ = 0.5, the arrival probabilities are exactly 40% above the random-time probabilities. In

fact, in the case of the hyper-exponential distribution, one can bring ξ as close to 1 as one

likes. The value of ξ is, as one can easily verify

ξ = (ρ + 0.5) −√

(ρ + 0.5)2 − d (40)

where

d = 2ρ(2ρp(1 − p) + p2 + (1 − p)2) .

14

Table 1: Ratios between Arrival and Random-Time Probabilities for 10-Server Queueing

Systems

Deterministic Hyper-exponential, p = 0.1

Number in system Number in system

ρ c, c + 1, . . . cρ cρ − 1 0 c, c + 1, . . . cρ cρ − 1 0

0.1 0.000454 0.820260 1.926767 1.926767 1.613658 1.314438 0.525117 0.525117

0.2 0.034886 0.885213 1.393626 2.354828 1.579063 1.234833 0.932683 0.337294

0.3 0.136274 0.915168 1.248396 2.633459 1.533333 1.184810 0.980752 0.270742

0.4 0.268388 0.932590 1.181225 2.842044 1.473791 1.150093 0.993807 0.244074

0.5 0.406376 0.944029 1.142590 3.009447 1.400000 1.123642 0.997401 0.231374

0.6 0.540405 0.952128 1.117512 3.149595 1.315861 1.101470 0.996999 0.224304

0.7 0.667138 0.958169 1.099924 3.270255 1.228571 1.081081 0.994174 0.219861

0.8 0.785787 0.963126 1.086911 3.208292 1.144766 1.060167 0.989072 0.216815

0.9 0.896555 0.970787 1.077289 1.927885 1.068184 1.035295 0.980678 0.214592

As p → 0, Cv → ∞, and d → 2ρ. For this limit of d, (40) yields ξ = 1. This means that most

arrivals are spaced so closely together that almost every customer arrives while the previous

customer is still waiting for service. Between these rushes, there are lulls with very little

traffic.

If h 6= 1, then it is obviously impossible that πi = hPi for all i. For instance, when arrivals

are deterministic, h < 1, that is, in all states i ∈ F , πi = hPi < Pi. This must be compensated

by having probabilities for πI > PI for some I ⊂ C. Similarly, for hyper-exponential arrivals,

πi > Pi for i ≥ 0. Consequently, for some I ⊂ C, πi < Pi. This is also shown in Table 1.

In particular, in the last column, we give the ratios between π∅ and P∅ for a 10-server queue

with equal service rates. In the case of deterministic arrivals, these values are significantly

above the one, and in the case of the hyper-exponential queue, they are significantly below 1.

The ratios h and π∅/P∅ give the extremes of the spectrum. However, there are many states

which show significantly different probabilities at random and arrival times. It was observed

that if all states showing the same number of customers in the system are combined, then the

differences decrease somewhat. In the case of deterministic arrivals, the ratio closest to one is

obtained by combining all states showing cρ customers in the system. In the case of hyper-

15

exponential arrivals, the ratio closest to one is reached for cρ − 1. In general, we observed

that the ratios are closest to one for the number in the system at or slightly below cρ.

Besides ξ, πC is important. Figures 1 and 2 give πC for ρ = 0.5 and ρ = 0.9, with the

0 5 10Number of Servers

0.00

0.05

0.10

0.15

0.20

0.25

Pro

babi

lity

D

E2

M

HE(0.1)

Figure 1: Plot of πC against c for ρ = 0.5

number of servers in the system varying from 1 to 10. It can be seen that the probability

of being in state C immediately preceding an arrival decreases rapidly with the number of

servers when ρ = 0.5. For ρ = 0.9, the decrease is only gradual. It is notable that for c > 1

and ρ = 0.5, πC is largest in the case of the hyper-exponential arrivals, and smallest in the

case of deterministic arrivals. For ρ = 0.9, however, this relationship is reversed. Note that

as c increases, πC decreases. In fact, for c = 10 and ρ = 0.5 and constant service times, πC is

negligible. Hence, for high values of c, and medium values of ρ, it is unlikely that all servers

are busy. The tables of Hillier and Yu [12] are consistent with this observation.

In the ordered entry rule, upstream servers have preference over downstream servers. This

means that there is uneven workload among the servers. In fact, in the D/D/c queue with

homogeneous service rates, only the first cρ servers are working, and all others are idle. When

service times are exponential, this evens out somewhat, as shown in Figure 3. The model

underlying this figure is a 10-server queue with a traffic intensity of ρ = 0.5. The figure

plots the probability of selecting server sk against k. The curve with the highest starting

16

0 5 10Number of Servers

0.00

0.05

0.10

0.15

0.20

Pro

babi

lity

D

E2

M

HE(0.1)

Figure 2: Plot of πC against c for ρ = 0.9

point corresponds to deterministic arrival. This curve still shows a point of inflection around

server scρ. The next curve represents a model with Erlang-2 arrivals, followed by a curve

with exponential arrivals, which is in turn followed by a curve with hyper-exponential arrivals

with p = 0.1. As one can see, the bias toward the upstream servers lessens as the coefficient

of variation between arrivals increases. Though it is not shown here, the bias toward the

upstream servers decreases with ρ. In fact, as ρ approaches 1, all servers are busy almost all

the time.

The uneven distribution of work among the servers has a significant effect if the service

rates are not equal. If the traffic intensity is low, then most of the work is done by the first

server, and the system performance essentially reflects its performance. As the traffic intensity

increases, the speed of the second server becomes important, and a further increase in the

traffic intensity will make the speed of the third server relevant, and so on. Generally, one

would expect that πC is minimal if the fast servers are upstream, and the slow servers are

downstream, and that πC is maximal if this order is reversed. With πC , B and Lq will also

decrease. Of course, ξ is not affected by the server arrangement in the ordered entry queue,

or, in the general setup, with the selection procedure as expressed by the qIJ .

If one considers the number in the system, then one has to add the expected number of

busy servers to Lq. This number is λE(D), and with poor server arrangement, respectively,

17

0 5 10Sever Selected

0.0

0.1

0.2

Pro

babi

lity

D

E2

M

HE(0.1)

Figure 3: Probability that sk serves arrival. c = 10, ρ = 0.5

poor server selection, this number will also increase. In fact, Rosberg and Makowski [18]

proved for the M/M/c queue with small arrival rates that it is best to allocate each arrival

to the fastest server, or even wait until a fast server becomes available. Their proof required

a considerable amount of mathematics. There is also a result of Yao [24], which, though

not applicable here, is worth mentioning. Yao assumes that there is no queue, and that all

customers not finding an available server are lost. He then considers the probabilities pk that

an arrival is served by sk, and he shows that if the fast servers are first, then all pk, k = 1, . . . c

are smaller than if the fast servers are last. In our case, such a strong order, involving all

servers, is impossible because in the absence of customer loss, the sum of all pk must be one.

What is relevant here is πC , which determines the expected number in the queue, and the

expected number in the system. The results of Yao, though interesting, do not relate to these

measures.

It is not our intent to determine an optimal selection rule, or, in the case of ordered entry

queues, an optimal server arrangement. However, it is of interest how much can be gained

through such an optimization. The arguments given above would suggest that optimization

is most profitable if the likelihood that some servers are idle is substantial, as is the case

when either ρ is low or c is high. Moreover, when arrivals are poorly spaced, as in the case of

hyper-exponential arrivals, then all servers tend to be busy during a spurt, and they tend to

18

Table 2: Probabilities of Delay in Two Server Systems.

ρ = 0.5 ρ = 0.9

Select Select

Distrib. Fast server Slow server Ratio Fast server Slow server Ratio

HE(0.1) 0.55988 0.58571 0.9559 0.93759 0.94124 0.9961

E1 0.32609 0.36207 0.9006 0.85222 0.86196 0.9887

E2 0.22047 0.26006 0.8478 0.81312 0.82634 0.9840

E5 0.14517 0.18338 0.7916 0.77671 0.79296 0.9795

E10 0.11868 0.15503 0.7655 0.76091 0.77838 0.9775

D 0.09210 0.12548 0.7340 0.74250 0.76130 0.9753

HE(0.1): Hyper-exponential arrivals with p = 0.1

Ek: Erlang-k arrivals

D: Deterministic arrivals

be idle during a lull, and this limits the scope for optimization. In summary, the difference

between the best and the worst arrangement is most pronounced when ρ is low, c is high and

Cv is low. This is borne out by Tables 2 and 3. The discipline assumed is ordered entry. In the

first arrangement, the faster server preceded the slower ones, and in the second arrangement,

the order was inverted. These two arrangements are presumably correspond to the best and

the worst possibilities, with all other arrangements lying somewhere between these extremes.

We considered systems with two servers, and systems with four servers. In the systems

with two servers, the service rate of server one was 20, and the service rate of server two was

10. In the four server systems, there were two servers with a service rate of 20, and two servers

with a service rate of 10. The four server systems were thus comparable to the two server

systems, except that both servers were doubled. The following interarrival time distributions

were used: the hyperexponetial distribution with p = 0.1 (HE(0.1)), the Erlang-k distribution

with k = 1, 2, 5 and 10 (Ek), and deterministic distribution (D). The arrival rates were chosen

to achieve a traffic intensity λ/µ of 0.5 and 0.9, respectively. Because of (33) and (32), the

ratios given in Table 2 and 3 are also valid when considering the average number in the queue

and the average wait.

19

Table 3: Probabilities of Delay in Four Server Systems.

ρ = 0.5 ρ = 0.9

Select Select

Distrib. Fast server Slow server Ratio Fast server Slow server Ratio

HE(0.1) 0.38437 0.42826 0.8975 0.89996 0.90783 0.9913

E1 0.15976 0.20652 0.7736 0.78437 0.80339 0.9763

E2 0.08788 0.12998 0.6761 0.73671 0.76119 0.9678

E5 0.04679 0.08063 0.5853 0.69407 0.72318 0.9597

E10 0.03461 0.06433 0.5380 0.67601 0.70698 0.9562

D 0.02372 0.04856 0.4885 0.65527 0.68829 0.9520

7 Computational Issues

This section addresses three different issues, namely the algorithm used to implement our

formulas, the performance of the algorithm, and the accuracy of the results. A program for

calculating the random and arrival time probabilities was written in C, using double precision,

and it was run on the DEC Alpha’s of the Department of Computer Science at the University of

Saskatchewan. The program contained 673 statements, and besides probabilities, it calculated

and printed many important expectations. At present, the program is restricted to ordered

entry queues.

Algorithmically, the main problem is the enumeration of all subsets. Avis [1] approached

a similar problem by using trees as data structures for representing the states. The resulting

algorithm is quite complex, and it uses a substantial amount of memory. We decided to rep-

resent the subsets of C by binary numbers with c digits. Specifically, if I ⊆ C, than server sk

is idle if the kth digit of I is 0, and sk is busy otherwise. The use of this representation makes

it easy to enumerate all states. For simplicity, we use I for both set I and its binary repre-

sentation. To enumerate all sets, one first sets I = 0 for the empty set, and one successively

increases I by 1 until I reaches C. Difficulties arise in the case where all subsets of I must

be enumerated. In our original program, we enumerated all numbers from 0 to 2n − 1, where

n = |I|, and then inserted zeros in the appropriate places. Later, we discovered a simpler

method for finding all subsets of I. Essentially, we skipped the digits at which I disappears

20

by using bit manipulations. Specifically, we used | for bit by bit disjunction, & for bit by bit

conjunction, and ∼ for bit by bit negation. One has

Theorem 1 To enumerate all subsets of I, let J = 0, and successively form

J := ((J | ∼I) + 1)&I .

This continues until I = C. Alternatively, one can set J = C, and form successively

J = (J − 1)&I .

This continues until J = 0, at which time the empty set is reached.

To prove this theorem, note that a binary number is increased by 1 by changing the lowest zero

digit to one, and by setting all digits to the right of this digit to 0. By forming (J | ∼I) + 1,

one essentially does the same, except that all digits for which I shows a zero are skipped.

The expression (J | ∼ I) + 1 has a one wherever I has a zero, and these ones are removed by

forming the conjunction with I. This proves the first method for enumerating all subsets of

I. The proof of the second method is left to the reader.

To insure program correctness, both unit testing and integration testing were conducted.

The routines for calculating L(s), L′(s), LK , ΦK and FK where tested individually, and

the results were checked against manual calculations. The procedure for calculating ξ was

based on Newton’s method, and this procedure was checked against the values published by

Şahin [20], and, in the case of the hyper-exponential distribution, against the formula given

in this paper. The method given in Theorem 1 was also given a unit test. The sums over

subsets found in (11), (18) and (28) were all implemented in one subroutine, with the arrays

L(µK), LK , ΦK and FK , respectively, passed as arguments. This allowed for more efficient

testing. For the solution of the equilibrium equations, the GTH method was used [9]. The

program implementing this method has been used extensively by the senior author for teaching

purposes, and no error was ever reported by students.

The outputs of the individual procedures allowed us to calculate the probabilities in ques-

tion manually. We also had an older program, which did not make use of Theorem 1 for

calculating the state probabilities, and which was also different in other aspects. In all cases,

the outputs matched. For special cases, the state probabilities can be obtained by alternative

21

methods. In particular, the formulas of the M/M/c queue with homogeneous service rates are

well known. The M/M/c queue heterogeneous service rates can be solved as a continuous-time

Markov chain. A program implementing both methods was written, and the results matched.

A sample of results involving Erlang and deterministic arrivals was also checked against the

tables of Hillier and Yu [12], and they agreed. Beside these independent evaluations, there

are a number of internal consistency checks. In particular, the random-time and arrival-time

probabilities must match in the case of the M/M/c queue, and this was the case. Another

internal check is given by (39), which was satisfied in all cases considered.

In fact, the correspondence between the results calculated by different methods was surpris-

ingly high. Normally, because of rounding errors, different methods lead to slightly different

results. Here, however, no discrepancies were discovered, except that once, the last digit

printed differed by a single unit. The printed output contained the five decimal digits in our

earlier runs. Later, six decimal digits were used.

Like the methods of Neuts [16], Yu [25] and Avis [1], our algorithm runs into performance

problems as c increases. The stumbling block is the solution of the system given by (18), which

requires one to solve 2c equations in 2c variables. Solving N equations in N variables requires

O(N3) operations. With N = 2c, this gives O(23c) = O(8c) operations. In other words, the

computational effort increases by a factor of 8 if c increases by 1 server. Hence, an eight-fold

increase in the speed of the computers only increases the largest problem solvable within a

given time frame by a single server. This means that even a large increase in computer speed

has only a marginal impact on the maximum size of the problem that can realistically be

solved. If solutions for systems with many servers are required, iterative method for solving

the equilibrium equations must be used. This, however, is beyond the scope of this paper.

For c = 20, even these methods become problematic. On a more positive note, we found that

the computer produced the results almost instantaneously for c ≤ 7. For c = 8, there was a

noticeable gap between the submission of the program and the printing of the results. This

wait increased considerable for c = 9. The calculations for c = 10 took around two minutes.

We did not try the program for c > 10.

Acknowledgement

The authors thankfully acknowledge the financial support of the NSERC, grant OR0008112

and OR0008185. They also thank the referees for their valuable suggestions.

22

References

[1] Avis, D. M., Computing Waiting Times in GI/Ek/c Queueing Systems. TIMS Studies

in Management Sciences 7, 215-232, 1977.

[2] Al-Khayyal, F. A. and Gross, D. Approximating and Bounding GI/M/c queues. TIMS

Studies in Management Science 7, 233-245. 1977.

[3] Cooper, R.B. Queues with Ordered Servers That Work at Different Rates. Opsearch 13,

394–403, 1976.

[4] Cooper, R.B. Introduction to Queueing Theory. North Holland 1981.

[5] Cooper, R.B. and Palakurthi, S. Heterogeneous Loss with Ordered Entry: An anomaly.

Operations Research Letters 6:8, 347-349, 1989.

[6] DeSmit, J. H. A. On Many Server with Exponential Service Times. Adv. of Appl. Prob.

5, 170-182, 1973.

[7] Disney, R. L. Some Multichannel Queueing Problems with Ordered Entry–An Application

to Conveyor Theory. J. Indust. Eng. 14(2), 105-108, 1963.

[8] Gnedenko, B.W. and Koenig, D. Handbuch der Bedienungstheory II. Academie-Verlag

1984.

[9] Grassmann, W. K., Taksar, Heyman, D. P. Regenerative Analysis and Steady State Dis-

tributions for Markov Chains. Operations Research 33, 1107-1116, 1985.

[10] Gross, D. and Harris, C. Fundamentals of Queueing Theory. Second edition, John Wiley

And Sons. 1985.

[11] Gumbel, H. Waiting Lines with Heterogeneous Servers. Operations Research 4, 504-511,

1960.

[12] Hillier, F. S. and Yu O. S. Queueing Tables and Graphs. Elsevier North Holland, New

York 1981.

[13] Matsui, M and Fukuta, J. On a Multichannel Queueing System with Ordered Entry and

Heterogeneous Servers. AIIE Trans. ((2), 209-214, 1977.

23

[14] Morse, P. M. Queues, Inventories and Maintenance. John Wiley, New York, 1967.

[15] Nawijn, W. M. A Note on Many-Server Queueing Systems with Ordered Entry, with

Applications to Conveyor Theory. J. Appl. Prob. 20, 144-152, 1983.

[16] Neuts, M. F. Matrix-Geometric Solutions in Stochastic Models, an Algorithmic Approach.

John Hopkins University Press, Baltimore, 1981.

[17] Pourbabai, B. and Sonderman, D. Service Utilization Factors in Queueing Loss Systems

with Ordered Entry and Heterogeneous Servers. J. Appl. Prob. 23, 236-242, 1986.

[18] Rosberg, Z. and Makowski, A. M. Optimal Routing to Parallel Heterogeneous Servers-

Small Arrival Rates. IEEE Transactions on Automatic Control, 35, 789-796, 1990.

[19] Rubinovitch, M. The Slow Server Problem: a Queue with Stalling. J. Appl. Prob. 22,

879-892, 1985.

[20] Şahin, I. On the Least Positive Root on the Equation z = K(z) in Queueing Theory.

Canadian Journal of Operational Research, 8, 109-115, 1970.

[21] Shantikumar, J. G. and D. D. Yao. Comparing Ordered-Entry Queues With Heterogeneous

Servers. Queueing Systems 2, 235-244, 1987.

[22] Singh, V. P. Markovian Queues with Three Heterogeneous Servers. AIIE Transactions 3,

45-48, 1971.

[23] Tákacs, L. Introduction to the Theory of Queues. Oxford University Press, New York

1962.

[24] Yao, D.D. The Arrangement of Servers in an Ordered-Entry system. Operations Research

35, 759-763, 1987.

[25] Yu, O.S. The Steady State Solution of a Heterogeneous-Server Queue with Erlang Service

Times. TIMS Studies in the Management Sciences 7, 199-213, 1977.

24