routing and stang in customer service chat …schwert.ssb.rochester.edu/tezcan/tz_wp13.pdfkey words:...
TRANSCRIPT
Routing and sta�ng in customer service chat systemswith impatient customers
Tolga Tezcan*University of Rochester, Simon Graduate School of Business
Jiheng Zhang †
The Hong Kong University of Science and Technology
We consider customer service chat (CSC) systems where customers can receive real time service from agents
using an instant messaging (IM) application over the Internet. A unique feature of these systems is that agents
can serve multiple customers simultaneously. The number of customers that an agent is serving, referred
to as the level of that agent, determines the rate at which each customer assigned to that agent receives
service. We consider the sta�ng problem in CSC systems with impatient customers where the objective is
to minimize the number of agents while providing a certain service level. The service level is measured in
terms of the proportion of customers who abandons the system in the long run. First we propose e↵ective
routing policies based on a static planning LP, both for the cases when the arrival rate is observable and for
when the rate is unobservable. We show that these routing policies minimize the proportion of abandoning
customers in the long run asymptotically for large systems. We also prove that the sta�ng solution obtained
from a sta�ng LP, when used with the proposed routing policies, is asymptotically optimal. We illustrate
the e↵ectiveness of our solution procedure in systems with small to large sizes via numerical and simulation
experiments.
Key words : Call Center Management, Queuing Theory and Stochastic Methods
History :
1. Introduction
Customer service chat (CSC) is a service provided by companies to allow users to communicate in
real time with a customer service agent using the Internet. It is based on an instant messaging (IM)
application that is usually accessible from a company’s web site. IM applications are text-based
one-on-one interaction tools, which allow individuals to communicate with each other while they
perform other computer-based tasks. The CSC service is usually a part of a company’s customer
contact center that also includes telephone and email customer service. In terms of responsiveness,
⇤ Research supported by NSF Grants CMMI-0954126 and CMMI-1130346.
† Research supported by Hong Kong RGC grant GRF-622411.
1
2
CSC service can be placed between telephone and email customer service. Unlike traditional email
service (which lacks immediacy and interactivity), it is real time. However, it is somewhat slower
compared to the telephone service since a user still has to wait for the agent to read each message
and type the response, see Koyama (1998) and Broughton (2001).
CSC systems have been shown to be operationally less costly than telephone customer service
support and about the same as email support, see Bannan (2000). Moreover, Shae et al. (2007)
demonstrates that a CSC system can perform better than a telephone service across di↵erent key
performance measures including first call resolution and end user satisfaction. Due to their unique
features, such as their ability to allow collaborative browsing or screen sharing, the CSC systems
are considered to be an especially important way to interact with customers for computer and
software companies, web based companies (such as retail e-commerce), and for libraries to provide
reference service, among others, see Francoeur (2001), Bannan (2000), and International (2011). In
addition, customers can carry out other tasks in their computers while the agent is trying to figure
out a solution for their problem. CSC systems also have some certain disadvantages such as the
technical proficiency required from customers, the lack of a natural conversation setting in a chat
environment, and the interaction being slowed down by the typing of messages, which leads to users
losing patience and sometimes abruptly terminating their chat sessions, see Koyama (1998) and
Oder (2001). However, there is a growing industry of companies providing software applications
for CSC systems, making them more accessible for customers and easier to manage for service
providers, see for example International (2011). Due to its popularity among the computer savvy
younger generation, see International (2011), its prevalence in customer service is only expected to
increase in the future.
As in other customer service systems, the quality of service is an important determinant of the
success of CSC systems, and better service quality helps overcome some of the aforementioned
obstacles in providing e↵ective chat service. In this paper we consider the sta�ng and routing
problems in CSC systems with impatient customers to provide good quality of service. The fact
that agents can serve multiple customers provides a unique challenge for the management of these
systems. The current literature on management of service systems mainly focus on systems in
which an agent only helps one customer at a time, see Gans et al. (2003). There is a vast literature
on processor sharing queues (where an agent can serve multiple customers simultaneously), which
can be found mainly in communications literature, see Zhang et al. (2009) and the references
therein. However, the CSC systems have di↵erent features that are not covered by this literature.
Specifically, the main assumption in this literature is that there is a single server (or a pool of
3
servers that work jointly) serving all or a subset of customers in the system. In a CSC system,
there is a pool of agents with limited parallel processing capability and each customer interacts
with the same agent until that customer leaves the system. Therefore, we will build new analytical
tools to analyze these systems.
Impact of multitasking on service systems and the routing problem: Before we study how to
manage CSC systems, it is important to highlight the e↵ect of multitasking on the productivity
of agents. Intuitively, chatting with more customers should increase the productivity of the agent
and it should also increase the time to complete service, but it is not clear to what extent. To the
best of our knowledge there is no comprehensive empirical study on the e↵ect of multitasking in
chat service systems. Shae et al. (2007) demonstrates that agents can handle three simultaneous
chat sessions without performance degradation (they recommend experimenting with more chat
sessions until there is performance degradation but they stop at three). Based on data from a
company that is operating a large IM-based service center, Luo and Zhang (2012) observe that the
total service rate of an agent is not even monotone in the number of customers the agent is helping.
They observe that the average total service rate becomes lower when an agent helps 4 customers
instead of 3 but it becomes higher than the service rate with 3 customers when the agent helps
5 agents (see Figure 1 there). Given that agents have di↵erent service rates at di↵erent levels of
multitasking, how to route the arriving customer to available agents can significantly a↵ect the
system performance. Thus, routing is quite an important decision to meet a certain service level
as well as minimizing the sta�ng level.
In general, the main advantage of mutitasking in a service system is that a multitasking agent
can switch between tasks whenever the current task the agent is doing is not available or when
some feedback is needed from the customer or another resource, eg. an agent waiting for a response
from a customer while providing chat service, a physician waiting for the result of lab tests in
an ER. However, the e↵ects of multitasking on the speed of carrying out a task can be di↵erent
in di↵erent contexts. Because of cognitive switching costs, it has been observed in experiments
that multitasking may lower productivity and quality in completing certain tasks (see Gladstones
et al. (1989) and Pashler et al. (1994)). Similar e↵ects have also been observed in service systems,
referred to as slow-down in Batt and Terwiesch (2013). Surprisingly, increased multitasking or
load may also increase the speeds of carrying out certain tasks, referred to as rushing, due to,
for example, increased pressure to complete the tasks. There are several papers which provides
empirical evidence that rushing exists in service systems; specifically Hasija et al. (2010) observes
rushing behavior of agents providing email service, Kc and Terwiesch (2009) of hospital employees
4
responsible for transporting patients, Tan and Netessine (2012) of servers in a restaurant chain,
and Staats and Gino (2012) of workers in a production simulation game.
In summary, there seems to be no universal trend in agent e�ciency while multitasking (and
so in service rates in chat-service systems). Due to slow-down the service rates tend to decrease
but due to rushing and reduced waiting for the customer responses the service rate may increase
with increased multitasking. In addition, if the quality of service and its e↵ect on service times are
taken into account (for example customers are unlikely to end a chat session until their problems
are resolved successfully) it is less likely that a universal trend is ubiquitous in all applications.
Therefore we analyze chat systems when increased multitasking may have a↵ects of varying degrees
on service times.
Our contribution: To incorporate the e↵ect of multitasking in our model we assume that the
service rate of an agent is determined by the number of customers, referred to as the agent’s level,
that agent is serving. Our main goal is to find the minimum sta�ng level in a CSC system to meet
a service level requirement given the arrival rate and service speeds. The service level is measured
in terms of the abandonment probability of a customer. En route to achieve this goal we first
identify conditions that determines what level of multitasking is e�cient (or ine�cient). These
conditions are based on the optimal solution of a static planning problem, which we refer to as the
routing LP. Given a fixed arrival rate and a sta�ng level, the routing LP determines how many
customers each agent should help on average in the long-run. We prove using the dual of this LP
that ine�cient levels should never be used, providing guidance on how to take advantage of the
multitasking capability of agents.
Although the routing LP provides a solution for the long-run allocation of agents into di↵er-
ent levels, it is not clear how CSC systems can be managed in real time (by allocating arriving
customers to available agents) to achieve such long-run performance. This becomes especially chal-
lenging when the arrival rate is not observable. We next focus on the routing problem to determine
which agent an arriving customer should be routed to. We propose two policies; one that assumes
the basic levels (levels that the routing LP allocates agents to) are known, and a second one that
is somewhat similar but estimates the basic levels seamlessly and automatically based on a virtual
system. The virtual system is similar to the CSC model but allows a certain type of preemption
which is not possible (or desirable) in actual CSC systems. We prove that the proposed policies
are asymptotically optimal in minimizing the probability of abandonment in steady state asymp-
totically for large systems when service and abandonment times are exponential. In addition, we
provide conditions for a simple static priority policy, referred to as lightest-load-first, that routes a
5
customer to the least busy agent to be asymptotically optimal. Finally, we propose a solution for
the sta�ng problem using another LP and prove that the sta�ng levels found by the sta�ng LP
combined with the proposed policies are asymptotically optimal.
Our solution approach is based on a “fluid limit” analysis similar to Atar et al. (2010, 2011),
Behzad and Tezcan (2012) and also to Bassamboo and Randhawa (2010), Bassamboo et al. (2010).
We consider a sequence of systems where the arrival rate and the number of servers get large
but the service time distribution and service level requirement remain fixed. We then analyze the
steady state of this system and characterize the abandonment probability. A critical part of our
analysis is proving that the fluid limits of CSC system under the proposed policies achieve the
desired steady state. We prove this result using carefully constructed Lyapunov functions.
We illustrate via numerical examples that the proposed routing algorithms and the sta�ng
solutions perform remarkably well in systems with sizes ranging from small to large. Even in systems
with optimal number of agents less than 10, our sta�ng LP misestimates the optimal solution by
at most 1 agent. In general, we show that the solutions found by the sta�ng LP are very close to
the best solutions found by a simulation search procedure, with a di↵erence less than 2 agents (or
less than 4%) across systems with di↵erent sizes. It is also interesting to note that the optimality
gap between these two solutions does not seem to grow with system size, reminiscent of the results
in Bassamboo et al. (2010), Bassamboo and Randhawa (2010). In Bassamboo et al. (2010) and
Bassamboo and Randhawa (2010) the fluid limit analysis has been shown to provide very accurate
sta�ng solutions for systems where agents can only serve a single customer at a time when the
arrival rate is random.
This paper is a part of authors’ continuing research e↵ort on analyzing CSC systems. In Luo and
Zhang (2012), CSC systems without abandonment are analyzed under the lightest-load-first policy.
They consider a cost function that depends on the level of customers in service and the number
of customers in queue. The decision variables are the sta�ng level and the maximum number of
customers an agent can serve. In this paper we extend their analysis in two important directions.
First we consider systems with impatient customers and second we identify asymptotically optimal
routing policies. We are also able to focus on steady state quantities. By considering more general
policies, not only are we able to identify e�cient and ine�cient levels but also to show when the
lightest-load-first is asymptotically optimal. In a follow up paper Tezcan and Zhang (2013), we
consider CSC systems with general service and abandonment times. We extend our analysis to
determine ine�cient and e�cient levels in that more general setting and identify the maximum
number of customers a server should serve when customers are impatient by extensively relying
6
on the results in this paper. Then we o↵er approximations for the system performance using fluid
limits.
The rest of this paper is organized as follows. In §2, we present the CSC queueing model and we
formulate the routing and the sta�ng problems and the asymptotic regime we consider. We present
the routing LP in §3 along with its optimal solution. In §4, we propose routing policies first for the
case when the basic levels are known and then when they are not known. We also prove that they
are asymptotically optimal in minimizing the probability of abandonment from the system for a
given sta�ng level. We turn our attention to the sta�ng problem in §5. We present the results of
our numerical experiments in §6 and conclude in §7.
2. Customer service chat model
This section consists of three parts. First we explain and motivate the analytical model we use
to analyze the CSC systems in §2.1. Then, we introduce the details of the routing and sta�ng
problems we consider in §2.2. Finally, we introduce the asymptotic regime we use in our analysis
in §2.3.
2.1. Model
Chat service systems: In a CSC system, a customer interacts with an agent by sending and
receiving written messages on an IM application in (almost) real time. An agent, on the other
hand, usually handles multiple customers simultaneously and sends replies to customers’ messages
in a certain order. The actual dynamics of this process are quite complex. Service of each customer
consists of a (random) number of back and forth sessions with the agent. Each such session consists
of the customer typing a message and then the agent typing a reply. Also each customer takes a
random amount of time to type a message and the length of the message may depend on other
factors, such as how long the chat service has been going on, how long the previous messages were,
and etc. How an agent prioritizes customer messages is not standard as well. It may depend on
the content of a message, the agent’s preference, or the customer’s assigned priority, among other
factors. Also customers leave when their service is completed and new customers join the chat
system changing each agent’s work load intermittently. As a result, modeling every detail of a chat
service session is quite complex and it is unlikely to lead to analytical insights. Instead we use a
processor sharing model as we explain next. Processor sharing models have been used extensively to
establish the performance of round-robin type polling policies (see for example Kleinrock (1976)).
7
Processor sharing model: In a processor sharing model, the customers being served by an agent
share the agent’s service capacity simultaneously. Therefore, the agent’s capacity is assumed to
be infinitely divisible. Although this is not exactly the case in service chat systems, due to the
complexity of the actual system and the similarity of these systems to a processor sharing model
(since agents serve customers in a manner similar to round robin), a processor sharing model seems
appropriate for customer chat service models. We explain the details of the processor sharing model
we use next.
We refer to the number of customers an agent is serving as that agent’s level. We assume that
each agent can serve up to I customers at the same time. If all the agents are serving I customers
at the time a customer arrives, the arriving customer joins the queue. An agent serving less than I
customers is said to be available. When a customer arrives to the system, that customer is assigned
to one of the available agents (as explained below) if there are any. Customers waiting in the
queue are served according to the first-come-first-serve policy. We assume that agents work in a
non-idling fashion: if they finish serving a customer and the queue is nonempty at that instant,
they start serving the next customer in queue. Also, once a customer is assigned to an agent, that
customer receives service only from that agent until the customer leaves the system (due to service
completion or abandonment). Similarly, an agent serves only those customers that are assigned
to him and preemptions or hand-o↵s are now allowed. Let µi
denote the rate a customer receives
service when an agent at level i is serving that customer. The total rate an agent at level i is
providing service, denoted by d
i
, is therefore equal to iµ
i
.
Each arriving customer requires a random amount of service. When the total amount of service a
customer receives reaches the customer’s service requirement, that customer leaves the system. We
assume that each customer’s service amount is exponentially distributed with mean 1, without loss
of generality since the time unit can be rescaled. Customers are assumed to arrive to the system
according to a renewal process with rate �, which is assumed to be time-independent.
We consider CSC systems where customers have limited patience. To model this behavior we
assume that associated with customer k there is a pair of independent exponential random variables
(⌫q
k
,⌫
k
). Random variable ⌫q
k
denotes the kth customer’s patience time while waiting in queue, that
is, if the kth customer’s waiting time in queue exceeds ⌫
q
k
, that customer abandons the system
immediately without receiving service. Random variable ⌫
k
denotes the kth customer’s patience
time while being served, that is, if the time in service for the kth customer exceeds ⌫k
, the customer
abandons the system immediately without finishing service. (We again emphasize the fact that the
time in service for a customer depends on the level of the agent serving that customer.) We assume
8
that {(⌫q
k
,⌫
k
)} is an i.i.d. sequence. We also assume that the sequences of patience, service and
interarrival times are mutually independent. Also ⌫
q
i
and ⌫
i
are assumed to be exponential with
rates � and ⌫, respectively.
Next we describe our assumptions on system parameters. First we define
d
i
= i(µi
+ ⌫), (1)
the total rate customers leave a level i agent and
P
Ab
i
= ⌫/(µi
+ ⌫) (2)
denotes the probability that a customer abandons the system if the customer is served at rate µ
i
.
When an additional customer is assigned to an agent, it’s likely that agent will not be able to
spend as much time with each customer as before since his e↵ort must be divided among more
customers now. Therefore, it is expected that µi
is decreasing in i. Hence we assume that
µ
i
>µ
i+1, for i= 1,2, . . . , I � 1. (3)
However, we expect that di
should be increasing in i until a certain level. This is due to the fact that
when there are more customers assigned to an agent, the agent will have to wait less for a customer
to send a new message. We observed in practice that the average time an agent spends to answer
each customer message increases as that agent handles more customers. This can be attributed to
the fact that the agent has to handle more tasks when there are more customers. After a certain
number of customers, it is not possible for an agent to handle all customers simultaneously in an
e↵ective way. Therefore, we assume that each agent can handle at most I customers.
Ine�cient levels: Next we present a classification of levels into two di↵erent classes which plays
a fundamental role in our analysis. A level i is said to be ine�cient if
d
i
< d
i
0 for some 1 i
0< i, (4)
or if there exists 1 i1 < i i2 I such that
d
i
i1 � i
i2 � i1d
i2 +i2 � i
i2 � i1d
i1 . (5)
Other levels are referred to as e�cient levels. It is easy to check that d’s can be replaced with d’s
in (5). Loosely speaking, (5) implies that instead of having an agent spend one unit of time at an
ine�cient level i, we can have the agent work a certain fraction of that time at level i1 and the
rest at level i2 to achieve a higher throughput without increasing the abandonment rate.
9
We let F = {i1, i2, . . . , iJ} denote the e�cient levels and F
c the ine�cient levels. We assume that
I 2F and that
�
1�P
Ab
I
�
d
I
= d
I
�
�
1�P
Ab
i
�
d
i
= d
i
, (6)
for all i I. It can be shown that if (6) does not hold it is optimal not to have agents in level I, a
detailed analysis is presented in Tezcan and Zhang (2013). Also, by definition 12F .
Let N denote the set of levels whose service rate is less than a lower level. That is
N =n
i : di
0> d
i
, for some i
0< i
o
.
Recall that by our assumption (6), I /2N . We also define
N
0 = {i : i+12N} .
2.2. Routing and sta�ng problems
An arriving customer may find multiple available agents at the time of arrival. We refer to the
manner in which customers are routed to available agents as the routing policy. One of our objec-
tives in this paper is to identify policies that minimize the probability of abandonment in steady
state and to establish approximations for the performance of such policies. Given a routing policy
⇡ let Ab
�,N
⇡
(t) denote the total number of abandonments from the system (queue and service) by
time t when the arrival rate is � and the number of agents is N . Given the arrival rate � and the
sta�ng level N we define P
Ab,⇡(�,N) by
P
Ab,⇡(�,N) = limT!1
E
Ab
�,N
⇡
(T )
T�
�
. (7)
The limit may not exist in general (or may depend on the initial state), but we will show below
that it exists and it is unique for non-idling policies. The quantity P
Ab,⇡(�,N) can be viewed as
the probability of abandonment of a customer who arrives at the chat system in steady state. For
the rest of the paper we restrict our attention to non-idling policies that are admissible, specifically
we only consider policies that are Markovian.
Another related operational objective is to determine how many agents to sta↵ the system with
to achieve a certain service level. The service level is measured in terms of the abandonment
probability from the system in the long-run. Formally, the sta�ng problem can be formulated as
follows: given � and p
Ab
> 0
min⇡,N
N s.t. PAb,⇡(�,N) p
Ab
. (8)
10
In this formulation the decision variables are the routing policy ⇡ and the sta�ng level N . The
sta�ng problem is closely connected to the routing decisions since the performance of a system
depends on how these decisions are made. The main focus of this paper is on the routing problem,
but we return our attention to the sta�ng problem in §5 once we present our solution for the
routing problem.
2.3. Asymptotic framework
Due to the complexity of the CSC systems it is not possible to carry out an exact analysis in
general. Even for relatively less complex call center systems, when available, exact solutions do
not provide much insight as to how to manage these systems. On the other hand, the asymptotic
analysis has been successfully used in complex systems to study scheduling and sta�ng problems
(see Gans et al. (2003) and Aksin et al. (2007)). The asymptotic analysis is also useful in finding
solutions that are easy to implement. In service systems, the many-server asymptotic analysis is
especially applicable since in most systems there is a pool of agents providing similar services to
customers (see Gans et al. (2003)). Hence, we use a many-server asymptotic analysis to provide
solutions for the routing problem when the system size is large. To identify asymptotically optimal
routing policies we consider a sequence of systems indexed by the arrival rate � and assume that
N
� =N�+ o(�). (9)
In applications, the size of the system is unlikely to a↵ect the characteristics of arriving customers
and the service rates. Therefore, we assume that the service and patience time distributions as well
as the service rates at each level are not dependent on �. A slightly di↵erent asymptotic regime is
introduced in §5 for the sta�ng problem.
3. Static planning problem
Static planning problems play a fundamental role in the analysis of queueing systems, see Harrison
(2000) and Williams (1998) among others. They provide insights into what the expected state of
a queueing system should be under an e↵ective control policy in the long run. In this section we
formulate a static planning problem for the optimal allocation of agents among all levels, referred to
as the routing LP. The objective of this LP is to minimize the probability of abandonment in a CSC
system with a fixed arrival rate and sta�ng level. Also, the definition of the e�ciency/ine�ciency
concept is based on the routing LP (or its dual). We show below that there should not be any
agents at ine�cient levels in the optimal solution of the routing LP. We also show at the end of
this section that the routing LP provides a lower bound for the asymptotic performance of any
routing policy. A closely related sta�ng LP will be presented in §5.
11
For fixed � and N consider
min{�i,i=1,...,I,I+1}
P
Ab (10)
st. �
I+1 +I
X
i=1
�
i
P
Ab
i
= �P
Ab
, (11)
I
X
i=1
�
i
d
i
N, (12)
I+1X
i=1
�
i
� �, (13)
�
i
� 0.
The objective of the routing LP is to minimize the probability of abandonment. The decision
variables �i
’s can be viewed as the average rate the customers are served by agents at level i, for
1 1 I and �
I+1 can be viewed as the rate customers abandon the queue. Constraint (11) is
an approximation for the probability of abandonment, based on (2). (In the following sections we
show that this approximation is asymptotically valid.) Constraint (12) states that �
i
’s must be
selected so as not to violate the capacity constraint, based on Little’s law (the asymptotic validity
of this claim will be established below). Constraint (13) implies that all arriving customers must
depart from the system. We use P
Ab (�,N) to denote the optimal objective function value.
Now we obtain a closed-form solution for the routing LP. (We will show that the solution of
the LP can be used to approximate the steady state of the proposed policies in the next section.)
Let F = {i1, i2, . . . , iJ} denote the ordered list of the indices of e�cient levels and i
⇤j+1 denote the
lowest indexed e�cient level such that
d
i
⇤j+1
� �/N. (14)
Lemma 1. If � d1, an optimal solution �
⇤ = (�⇤i
, i= 1, . . . , I) of the routing LP (10) is given
by
�
⇤1 = �, and �
⇤i
= 0, for i 6= 1. (15)
If dI
N > �> d1 an optimal solution of (10) is given by
�
⇤i
⇤j=
d
i
⇤j
d
i
⇤j+1
� d
i
⇤j
⇣
Nd
i
⇤j+1
��
⌘
, �
⇤i
⇤j+1
= ���
i
⇤j=
d
i
⇤j+1
d
i
⇤j+1
� d
i
⇤j
⇣
��Nd
i
⇤j
⌘
, (16)
and �
⇤i
= 0 for i 6= i
⇤j
, i
⇤j+1. If �� d
I
N , an optimal solution of (10) is given by
�
⇤I
= d
I
N, �
⇤I+1 = �� d
I
N and �
⇤i
= 0, for i 6= I, I +1. (17)
12
The proof is presented in Appendix A. Because di
⇤j+1
> d
i
⇤jby (4), the solution given by (16) is well-
defined. Denote the optimal solution described in Lemma 1 by �
⇤ (�,N) = (�⇤i
(�,N) ; i= 1, . . . , I).
In general there may be alternative optimal solutions of the routing LP (10), but from here on we
only consider the optimal solution �
⇤ (�,N). We define
z
⇤i
(�,N) = �
⇤i
(�,N)/di
, i= 1, . . . , I (18)
and z
⇤0 (�,N) =N �
P
I
i=1 z⇤i
(�,N). Also, we set z
⇤ (�,N) = (z⇤i
(�,N) , i= 0,1, . . . , I) and refer to
levels in the set i⇤(�,N), {i : z⇤i
(�,N)> 0} as basic levels.
We highlight the fact that there can be at most two basic levels by Lemma 1. In addition, an
ine�cient level can never be basic. This motivates the our classification of states into two classes;
e�cient and ine�cient. The intuition behind this result can best be explained by considering
the case d
I
N > � > d1. In words, this result implies that in the optimal solution, the agents are
distributed between two e�cient levels i⇤j
and i
⇤j+1 with lowest indexes such that the total service
rate can be made equal to the arrival rate. Because the abandonment probability is increasing in
the index for e�cient levels, the feasible solution with the lowest indexed e�cient levels is optimal.
We next show that the optimal solution of the LP provides an asymptotic lower bound for the
abandonment probability under any policy.
Theorem 1. Consider a sequence of chat service systems that satisfies (9). Then for any
sequence of admissible policies {⇡
�
} we have
lim inf�!1
P
Ab,⇡
�(�,N�)� P
Ab(1,N), (19)
where P
Ab(1,N) is the optimal value for the routing LP (10).
The proof is placed in Appendix C.
4. Routing policies
In §3 we proposed a static planning problem approach to obtain a solution for the routing problem
in order to minimize the probability of abandonment. In CSC systems, arriving customers are
assigned to one of the available agents at the time of their arrivals and it is not clear how one can
dynamically manage such a system to have the long run behavior given by the solution of (10).
In this section we propose routing policies to achieve the long-run proportions suggested by the
solution of Lemma 1 for fixed arrival rate and number of servers. In §4.1 we consider the case when
the basic levels are known and we present some special cases when the proposed control policies
can be simplified in §4.2. We treat the case when the basic levels are not known in §4.3.
13
4.1. When basic levels are known
In this section we consider the case when the basic levels ij1 and i
j2 of the optimal solution z
⇤(1,N)
are known, with the possibility that i
j1 = i
j2 . (We drop ⇤ from the notation for simplicity.) Let
U
ij= {i
j
+ 1, . . . , ij+1 � 1} if i
j
+ 1 < i
j+1 and ; otherwise, denoting the set of ine�cient levels
between e�cient levels ij
and i
j+1, if there are any. Consider the following policy; at the time of a
customer arrival, let i denote the lowest nonempty level,
• if i < i
j1 , route the arriving customer to an agent serving i customers.
• if ij1 i < i
j2 , route the customer to an agent in the highest nonempty level in U
ij1. If all the
levels in U
ij1are empty or if U
ij1is empty, route the customer to an agent in level i
j1 .
• if ij2 i < I route the customer to an agent in the lowest nonempty level i0 /2N
0\{I}. If there
is no such agent, route the customer to an agent in the highest nonempty level in N
0.
A customer cannot be routed to an agent at level I. If all the agents are at level I, then the
customer joins the queue. We denote this policy by ⇡
�,⇤ (We drop � from the notation when � is
obvious from the context.) When there is only one basic level, ij1 , we follow the same policy but
there is no need for the second step, we only consider the first case, i < i
j1 , and the third case,
i� i
j1 . We emphasize the fact that in order to implement ⇡�,⇤, ij1 and i
j2 must be specified.
Example: We note that for fixed i
j1 and i
j2 , the proposed policy is a fixed priority policy. To
illustrate the application of this policy, consider a system with I = 8. Let ij1 = 1 and i
j2 = 3, hence
U
ij1= {2}. Also assume that N = {5,7} and so N
0 = {4,6}. Then, the priorities of the levels under
the proposed policy are given in the following order; 0,2,1,3,5,7,6,4.
Before we explain the intuition behind the proposed policy of ⇡�,⇤ we focus on its asymptotic
performance. Clearly, the set i
⇤(�,N�) of basic levels depends on �. Consequently, so does the
proposed policy. We now show that the proposed policy is asymptotically optimal and that it is
possible to obtain closed-form approximations for the performance of this policy using the optimal
solution of the routing LP. We use z
� = (z�1 , z�
2 , . . . , z�
I
) to denote the number of agents at level i
in steady state under the proposed policy ⇡
�,⇤ and define z
� = z
�
/�.
Theorem 2. Consider a sequence of chat service systems that satisfies (9). Then for the
sequence of proposed policies {⇡
�,⇤} we have
lim�!1
P
Ab,⇡
�,⇤(�,N�) = P
Ab(1,N), (20)
in addition
z
�
) z
⇤(1,N), as �!1. (21)
It follows from this result and Theorem 1 that the sequence of policies {⇡�,⇤} is an asymptotically
optimal routing policy, see Appendix D for a proof.
14
Discussion on the proposed policy: The proposed policy aims to have all agents in two e�cient
levels ij1 and i
j2 as suggested by Lemma 1. By giving priority to levels below i
j1 , it aims to have
all the agents in level ij1 and above. However, if the lowest non-empty level is greater than or equal
to i
j1 giving priority to the lowest level may force the system to end up in a non-optimal steady
state. By setting the priority of levels in U
ij1increasing in their index, the policy aims to “empty”
the ine�cient levels starting from the one with the highest index in U
ij1. (It is easy to come up
with examples that if the priorities are in decreasing in index, the steady state will be suboptimal.)
The reason the priorities of levels in N and N
0 are set that way is similar. But now the issue is
that because levels in N may have a lower total service rate than levels with lower index it may
not be possible to have agents to go to lower levels if the priorities are not altered. By giving lower
priorities to levels in N
0, the policy manages to empty levels in N first. Then, the levels in N
0
becomes empty after a certain time.
One might question the necessity of switching the priorities of levels in the manner described or
the advantage of using the proposed policies over the simpler policies, such as the more intuitive
lightest-load-first policy. It is not very di�cult to come up with examples when not switching the
priorities in U
ij1and/or N and N
0 would result in suboptimal performance. However, in certain
situations simpler policies may yield asymptotically optimal performance. We discuss two such
cases in the following section. To what extent using a complex policy rather than a simple policy
improves the performance depends on the system parameters. In §6 we illustrate the di↵erence in
the performances of ⇡�,⇤ and the lightest-load-first policy in a set of numerical experiments. In
general, simpler policies fail to avoid having agents in ine�cient levels, hence the improvement in
performance by using ⇡
�,⇤ over such policies depend on how ine�cient the levels that need to be
avoided are.
4.2. Special Cases
In this section we discuss two special cases when the proposed policy can be simplified significantly.
First, if all the levels are e�cient, i.e., i2F for all i= 1, . . . , I, the solution to the routing problem
simplifies significantly. (It can easily be checked that if di
is concave as a function of i, then all
levels are e�cient.) If i2F for all i= 1, . . . , I, then i
j1 +1= i
j2 (if there are two basic levels), and
N = ;. Hence, the policy ⇡
⇤ becomes a static priority policy that gives priority to the lowest level
(we denote it by ⇡). Also, there is no need to estimate the basic levels i
j1 and i
j2 in this case.
Actually the lightest-load-first policy is asymptotically optimal if N and U
ijare both empty.
More insight can be gained about the lightest-load-first policy by considering another parsimo-
nious policy. Consider the following policy suggested by an anonymous referee; assign the incoming
15
customer to an agent to maximize the total service rate in the system, i.e., assign the customer to
level i that maximizes di+1�d
i
. When all the levels are e�cient and level “0” has priority over all
the other levels, this is the same policy as the lightest-load-first. To see this it is enough to show
that
d
i+1 � d
i
� d
i
� d
i�1, i= 1, . . . , I � 1 (22)
if all levels are e�cient. If (22) does not hold for some i then
2di
d
i+1 + d
i�1.
hence i cannot be e�cient by (5). Therefore, the lightest-load-first policy can be viewed as a
greedy policy which tries to maximize the instantaneous total service rate at all times (when all
the levels are e�cient). When ine�cient levels are present, lightest-load-first policy will obviously
not maximize the total service rate.
The second special case we consider is when
d1 < d2 < . . . < d
I
. (23)
If (23) holds, it is again possible to use a simpler policy (without needing to estimate the basic
levels). Note that in this case N = ;. Specifically, we propose the following policy. At the time of
a customer arrival, let i denote the lowest nonempty level and i
j
denote the e�cient level with
the highest index below i or set i
j
= i if i is e�cient. Then, route the customer to an agent in
the highest nonempty level in U
ij. If all the levels in U
ijare empty or if U
ijis empty, route the
customer to an agent in level ij
. If all the agents are at level I, then the customer joins the queue.
We denote this policy by ⇡
0.
One of the di↵erences between ⇡
�,⇤ and ⇡
0 is that in implementing ⇡
0 one does not need to know
the basic levels ij1 and i
j2 . Instead, the priorities of all ine�cient levels are changed. As an example,
again consider a CSC system with I = 8 and assume that levels 2, 4, and 5 are ine�cient. Then
under ⇡0 the priorities of the levels are given in the following order; 0,2,1,5,4,3,6,7. (Recall that
under ⇡�,⇤ the priorities depend on the basic levels hence on the arrival rate.) The following result
establishes the optimality of ⇡0 when (23) holds.
Theorem 3. Consider a sequence of chat service systems that satisfies (9) and (23). Then for
the sequence of proposed policies {⇡
0} we have
lim�!1
P
Ab,⇡
0(�,N�) = P
Ab(1,N), (24)
in addition
z
�,⇡
0) z
⇤(1,N), as �!1. (25)
16
4.3. Routing policies: When basic levels are not known
In this section we focus on the case when the basic levels are not known. For two special cases
we already discussed how the proposed policy can be simplified and basic levels do not have to
be estimated in §4.2. Next we treat the general case and we propose a policy that consists of two
components; the virtual system that is used to identify basic levels and the actual system where the
actual routing is done. Before introducing the virtual systems, we need to introduce the preemptive
systems which form the foundation of the virtual systems.
Preemptive Systems: Consider a system where preemption in the manner we explain next is
allowed. First note that Uij= {i
j
+1, . . . , ij+1 � 1}, if i
j+1 > i
j
+1, and U
ijis an empty set otherwise.
If there is an agent in one of the levels Uij(assuming it is not empty), refer to it as ine�cient-level
agent (we argue below that there can be at most one agent in levels U
ijunder the preemptive
regime). Assume that an agent in level ij+1 finishes serving a customer at time t, call this agent the
service-completion agent. At the instant the agent at level ij+1 finishes service, one of the customers
the ine�cient-level agent is serving is handed o↵ (whose service is said to be preempted) to the
service completion agent at this instant. Therefore, the level of the service completion agent does
not change and the ine�cient-level agent (who hands o↵ the customer) goes one level below. If
there are no agents in one of the levels in U
ijor if U
ijis empty, no action is taken.
Except for the explained preemptions, the policy we use in this setting is similar to that we
proposed in the previous section. Assume that, at the time of a customer arrival, i is the lowest
level with agents. If i /2F , route the customer to this level. If i2F , route the customer to the agent
in the highest level in U
i
, if Ui
is not empty and there is an agent in one of those levels. Otherwise,
route the customer to level i. We denote this policy by ⇡
P
. We note that this policy does not use
any information about the basic levels.
Now we argue that there can be at most one agent in one of the levels in U
ij, for any i
j
2F , if
customers are rearranged properly at time 0. Assume that at time zero there is more than one agent
in levels in U
ij( 6= ;). The customers that are being served by these agents can be redistributed
easily among these agents to make sure there is only one such agent left in one of the levels in
U
ij. For example, an agent at level i
j
+ 1 can hand-o↵ one customer to another agent in one of
the ine�cient levels, if there is such an agent, and goes to level ij
and the other agent’s level is
increased by one. Once the customers are re-arranged at time zero, there can be at most one agent
in a level in U
ijunder our preemptive policy.
We next show that ⇡P
identifies the correct allocations of agents among all the levels in the long-
run in the preemptive systems. First, we need to make certain assumptions on the initial states of
17
the systems to obtain meaningful limits. Let Z�
i
(t) denote the number of agents in level i and Q
�(t)
denote the number of customers waiting in queue at time t. We set Z
�(t) = (Z�
0 (t), . . . ,Z�
I
(t)),
Z
�(t) =Z
�(t)/�, and Q
�(t) =Q
�(t)/�. We assume that
lim�!1
Z
�(0) = Z(0), a.s., (26)
for some vector Z(0) =�
Z0(0), . . . , ZI
(0)�
and
lim�!1
Q
�(0) = 0, a.s. (27)
Proposition 1. Consider a sequence of chat service systems that satisfies (9), (26) and (27).
Under ⇡
P
limT!1
lim�!1
Z
�(T )! z
⇤(1,N) a.s. (28)
See Appendix E for a proof.
Non-preemptive systems with unknown basic levels: Although the policy ⇡
P
we proposed for
the preemptive systems identifies the basic levels automatically, in CSC systems preemption and
handing-o↵ are rarely used. This is mainly due to the fact that handing o↵ a customer to another
agent may lead to ine�ciencies since the agent who receives the customer does not know the history
of the chat session with the customer. Hence that agent is likely to spend extra time to go through
the conversation of the customer with the previous agent or ask the same questions again, which in
addition is likely to frustrate the customer. Therefore, we propose asymptotically optimal policies
that do not require preemption (or handing o↵ customers).
As mentioned above, we use a virtual system to identify basic levels. In this virtual system, the
number of (virtual) agents is the same as the actual system and so is the arrival process, but service
times are di↵erent. At each customer arrival we generate a random service time for the arriving
customer in the virtual system using the same service time distribution in the actual system. The
virtual system runs in the same manner as the preemptive system we described in the previous
section. The customers are rearranged in the virtual system at time zero as described above and
an arriving customer is routed to one of the virtual agents following ⇡
P
.
Next we describe the routing policy for the actual system that uses the virtual system. We
basically use the policy ⇡
⇤ proposed in §4.1 but we estimate i
j1 and i
j2 using the virtual system
as follows. Fix ✏> 0. If there are at least two e�cient levels in the virtual system with at least N✏
agents at time t, we use the e�cient level with the lowest index, denoted by i
j1(t) as an estimator
for ij1 and we use i
j1+1(t) as an estimator for ij2 . If there is only one such level, denoted by i
j1(t),
18
we use ⇡
⇤ with basic levels i
j1(t) and j1(t), where j1(t) is the smallest indexed level greater than
i
j1(t) that is not in N , if ij1(t) 6= I. If i
j1(t) = I, we use ⇡
⇤ with a single level. Aside from this
estimation procedure, ⇡⇤ is used as described in §4.1. That is, if a customer arrives at time t, we
use i
j1(t) and i
j1+1(t) (or j1(t)) instead of ij1 and i
j2 , respectively, in §4.1. The virtual system is
run simultaneously with the actual system. We denote this policy by ⇡
⇤ or by ⇡
⇤(✏) when we would
like to make the dependence on ✏ explicit.
The policy ⇡
⇤ is not an asymptotically optimal routing policy in the sense similar to (20) because
it turns out that it does not satisfy (28) in general. This is due to fact that in our estimation
procedure, a level is deemed basic if there are more than N✏ agents in that level in the virtual
system. On the other hand, it is possible that in the optimal solution of (10) a level is basic with less
than N✏ agents. However, since ✏> 0 is arbitrary, the performance of ⇡⇤(✏) can be made arbitrarily
close to the performance of ⇡⇤. We state this result next.
Theorem 4. Consider a sequence of chat service systems that satisfies (9). For � > 0, there
exists ✏> 0 such that the sequence of chat service systems under the policy ⇡
⇤(✏) satisfies
limsup�!1
P
Ab,⇡
⇤(�,N�) P
Ab(1,N)+ �, (29)
for any �� 0 (i.e. the choice of ✏> 0 does not depend on �). Also, if z⇤i
(�,N)> ✏ for i= i
⇤j
and
i
⇤j+1
limsup�!1
P
Ab,⇡
⇤(�,N�) = P
Ab(1,N), (30)
See Appendix F for a proof.
5. Sta�ng problem
In this section we turn our attention to the sta�ng problem (8). Recall that our objective is to iden-
tify the minimum sta�ng level that satisfies a service level constraint in terms of the abandonment
probability for a fixed arrival rate. We already identified policies to e↵ectively route customers to
available agents hence we restrict our attention to these policies. Our solution procedure therefore
is based on an LP, referred to as the sta�ng LP which is similar to the routing LP.
Sta�ng LP: Given the arrival rate � and the service level constraint pAb, consider the following
sta�ng LP.
min{N,�i:i=1,...,I+1}
N (31)
st.
I
X
i=1
P
Ab
i
�
i
+�
I+1 p
Ab
�, (32)
19
I
X
i=1
�
i
d
i
N, (33)
I
X
i=1
�
i
+�
I+1 � �, (34)
�
i
� 0, i= 1,2, . . . , I +1
The objective of the sta�ng LP is to minimize the number of agents. The decision variables (�i
’s)
have the same interpretation as in the routing LP with �
I+1 denoting the rate customers abandon
the queue. The left hand side of the constraint (32) can be understood as the total abandonment
rate for the system. The constraints (33) and (34) are identical to (12) and (13), respectively. The
sta�ng and the routing LPs are closely related as described in the following result. We denote the
optimal solution of the sta�ng LP (31) by N(�, pAb
). If pAb
<P
Ab
1 , the sta�ng LP is infeasible so
we assume that pAb
� P
Ab
1 . Let i⇤j+1 denote the lowest indexed e�cient level with
P
Ab
i
⇤j+1
� p
Ab
. (35)
Lemma 2. If PAb
1 p
Ab
<P
Ab
I
, an optimal solution (�⇤,N
⇤), with �
⇤ = (�⇤i
: i= 1, . . . , I +1), of
(31) is given by �
⇤i
= 0, if i 6= i
⇤j
, i
⇤j+1,
�
⇤i
⇤j=
P
Ab
i
⇤j+1
� p
Ab
P
Ab
i
⇤j+1
�P
Ab
i
⇤j
�, �
⇤i
⇤j+1
= ���
⇤i
⇤j=
p
Ab
�P
Ab
i
⇤j
P
Ab
i
⇤j+1
�P
Ab
i
⇤j
�, (36)
and
N
⇤ =�
⇤i
⇤j
d
i
⇤j
+�
⇤i
⇤j+1
d
i
⇤j+1
.
If pAb
� P
Ab
I
, an optimal solution of (31) is given by �
⇤i
= 0, i= 1,2, . . . I � 1,
�
⇤I
=�(1� p
Ab)
1�P
Ab
I
, �
⇤I+1 = ���
⇤I
, and N
⇤ = �
⇤I
/d
I
. (37)
The proof is very similar to that of Lemma 1, hence it is omitted. We note again that PAb
i
⇤j+1
>P
Ab
i
⇤j
by Lemma 3 hence (36) is well defined.
Asymptotic optimality: Now we show that the optimal solution of the sta�ng LP is asymptoti-
cally optimal when the arrival rate is large. For this purpose we use a slightly di↵erent asymptotic
regime from that we described in §2.3, we consider a sequence of systems where the arrival rate
becomes large but the service level pAb
> 0 and other system parameters are held constant. Consider
a sequence of systems indexed by the sequence arrival rates {�n
} such that
�
n
n
! � as n!1. (38)
20
A sequence of sta�ng levels {N
n
} is said to be asymptotically feasible if there exists a sequence
of policies {⇡n
} such that
limsupn!1
P
Ab,⇡
n(�n
,N
n) p
Ab
. (39)
Also, a sequence of asymptotically feasible sta�ng levels {N
⇤,n} is asymptotically optimal if for
any sequence of asymptotically feasible sta�ng levels {Nn
},
limsupn!1
N
⇤,n
N
n
1. (40)
Given the arrival rate �
n and service level constraint pAb, the sta�ng LP not only determines the
optimal sta�ng level N⇤(�n
, p
Ab) but also the basic levels, those levels with �
⇤i
> 0. It is easy to
show that these basic levels are those given by i
⇤(�n
,N
⇤(�n
, p
Ab)) (recall the definition given in
§3). Hence the proposed policy can be used in this setting as well with these basic levels. (In the
implementation of our policy we assume that the basic levels are known. We consider the case when
they are not below.) We denote this policy in the nth system by ⇡
n,⇤ for notational simplicity. We
next show the sta�ng LP (31) gives an asymptotically optimal sta�ng level when used with the
proposed dynamic priority policy ⇡
n,⇤.
Theorem 5. Consider a sequence of chat service systems that satisfies (38). The sequence of
sta�ng levels {N
⇤(�n
, p
Ab)} is asymptotically feasible when used with the sequence of proposed
policies {⇡
n,⇤}. Moreover, {N⇤(�n
, p
Ab)} is asymptotically optimal.
See Appendix G for a proof.
Unknown arrival rate: Finally, we consider the case when the basic levels are not known. This
case is especially important when the exact arrival rate is not known and sta�ng decisions are
needed to made based on the distribution of the arrival rate. In this case, there is a small ine�ciency
(relative to the case when the basic levels are known) in the system introduced by the estimation
procedure. However, the magnitude of the ine�ciency can be made arbitrarily small as we explain
next.
Based on Theorem 4, the proposed policy ⇡
⇤(✏) can be o↵ from the optimal solution by ✏.
This happens only when one of the basic levels should have less than ✏N agents in steady state.
Otherwise, ⇡⇤(✏) attains the same steady state as ⇡n,⇤. Therefore, it is possible to inflate the sta�ng
level N⇤(�n
, p
Ab) by a small amount � to obtain asymptotically feasible sta�ng solution. Next we
formalize these results.
21
Theorem 6. Consider a sequence of chat service systems that satisfies (38). Given �> 0, there
exists ✏> 0 such that the sequence of sta�ng levels {(1+ �)N⇤(�n
, p
Ab)} is asymptotically feasible
when used with the routing policy ⇡
⇤(✏).
See Appendix G for a proof.
6. Numerical Experiments
In this section we present the results of numerical experiments to illustrate the e↵ectiveness of the
routing policies we proposed and our solution procedure for the sta�ng problem. Throughout the
experiments in this section we consider a CSC system with six levels. We use the following service
rates for each level;
µ= {2.8,2,1.6,1.5,1.15,1.15},
therefore
d= {2.8,4,4.8,6,5.75,6.9}.
It is easily checked that only levels 3 and 5 are ine�cient (F = {1,2,4,6}, F c = {3,5}) and N = {5}.
We also set the abandonment rates ⌫ = � = 0.2. We first focus on the routing policies and assume
that the sta�ng level is fixed. We run experiments in six di↵erent settings with various arrival
rates and sta�ng levels. In Table 1, we display the associated data with each setting along with
the optimal solution z
⇤ = (z⇤1 , . . . , z⇤6) to the routing LP (10), see (18). (The sizes of the systems in
the third and sixth experiments are not very realistic and we use them to verify the asymptotic
accuracy of our results). We present the optimal abandonment probability estimated by LP (10)
under the column P
ab
for each setting as well.
We present the simulation results under three policies ⇡
⇤ (we drop � from the notation for
simplicity), ⇡⇤, and the policy that gives priority to the lowest nonempty level, denoted by ⇡.
(Recall that the policy ⇡ is asymptotically optimal if F c = ;.) Observe that, under the policy ⇡
⇤,
the priorities are given by (0,1,3,2,5,4) in the first three settings in Table 1 and by (0,1,2,3,5,4) in
settings 4–6. For the policy ⇡
⇤, we set ✏= 0.1. In each simulation experiment, we run the simulation
to allow 1.5 million arrivals and we use the first 20% of the simulation time as the warmup period.
The results of the simulations are presented in Table 2 for settings 1–3 and in Table 3 for settings
4–6. In these tables, under column Z
i
we present the average number of agents in level i and under
column P
ab
we present the percentage of customers that abandoned the system. The confidence
intervals in all the experiments are less than .5% of the mean values, therefore we do not include
them in our tables.
22
As expected from our asymptotic results, the performances of the systems under the proposed
policies ⇡
⇤ and ⇡
⇤ converge to the values estimated by the optimal solution of the routing LP
(10), presented in Table 1, as the system size gets large. In addition, the proposed policies perform
remarkably well even when there are only 25 agents. Also, the policy ⇡
⇤, which uses the virtual
system to estimate basic levels, performs slightly worse than the policy ⇡
⇤ in all cases, but the
di↵erence is very small. When there are 25 agents, the average di↵erence between the performance
of ⇡⇤ and the optimal solution of LP (10) is less than 4%. This di↵erence goes below 2% when
there are 250 agents. This shows that the static planning LP (10) is quite accurate in estimating
the system performance.
The di↵erences between the performance of ⇡⇤ and the optimal solution of the routing LP (10)
can be attributed to the fact that the occupancy in some levels are 0 in the optimal solution but
this cannot be achieved in reality without preemption. For example, in setting 2, there are 25
expected agents in level 4 and zero agents in level 3 in the optimal solution of (10). However, when
an agent completes service in level 4, that agent will have to wait for an arrival to go back to level
4 or complete another service to go to level 2, the other basic level. In either case, the agent will
have to wait for a certain amount of time because the arrivals (or service completions) are not
instantaneous and there may be other agents awaiting arrivals in non-basic levels 1 or 3. As the
system size gets larger, the wait will be shorter. This is also observed in our numerical experiments.
In setting 1, where the arrival rate is 140, the average number of agents in level 3 is around 10% of
all agents and when the arrival rate is increased to 1400, in setting 3, it decreases to 3.4%, under
⇡
⇤.
We present the results under ⇡ in order to compare our policies to another policy that does not
require the solution of the LP (10). In terms of the abandonment probability, it can be perform
as much as 13% worse than the proposed policies and its average performance in all experiments
is around 6% worse than ⇡
⇤. In order to have a better idea for the di↵erences between these two
policies (⇡⇤ and ⇡), we next simulate the system with N = 100 and arrival rate ranging from 400
to 800. We present the increase in the abandonment probability vs. the arrival rate when policy
⇡ is used instead of ⇡⇤ in Figure 1. We note that when the arrival rate is between 440 and 680,
the basic levels are 2 and 4 and between 680 and 800, they are 4 and 6. Note that ⇡
⇤ tries to
avoid having agents in levels 3 and 5 as much as possible. Also, level 5 is more “ine�cient” than
level 3, in the sense that using levels 4 and 6 instead of level 5 is more beneficial than using levels
2 and 4 instead of level 3. Hence, the di↵erence between the performances of these two policies
23
Setting � N z
⇤1 z
⇤2 z
⇤3 z
⇤4 z
⇤5 z
⇤6 P
ab
1 140 25 0 12.5 0 12.5 0 0 10.4%2 280 50 0 25 0 25 0 0 10.4%3 1400 250 0 125 0 125 0 0 10.4%
4 180 25 0 0 0 17.30 0 7.70 12.8%5 320 50 0 0 0 34.60 0 15.40 12.8%6 1800 250 0 0 0 173 0 77 12.8%
Table 1 Data for experiments and optimal solutions
Setting Policy Z0 Z1 Z2 Z3 Z4 Z5 Z6 P
ab
1 ⇡
⇤ 0.012 0.554 10.096 2.507 11.668 0.16 0.002 10.80%1 ⇡
⇤ 0.012 0.529 9.693 3.278 11.328 0.154 0.006 10.82%1 ⇡ 0.004 0.179 3.581 14.729 6.297 0.207 0.002 11.08%
2 ⇡
⇤ 0.006 0.573 21.859 3.707 23.831 0.024 0 10.75%2 ⇡
⇤ 0.006 0.571 21.767 3.874 23.755 0.026 0 10.76%2 ⇡ 0.001 0.101 4.758 34.323 10.78 0.038 0 11.12%
3 ⇡
⇤ 0.001 0.607 118.697 8.614 122.08 0 0 10.72%3 ⇡
⇤ 0.001 0.604 118.68 8.622 122.092 0 0 10.73%3 ⇡ 0 0.022 6.466 202.582 40.93 0 0 11.18%
Table 2 Results of simulation experiments: Settings 1–3
is significantly larger for higher values of the arrival rate. In general, the di↵erence between the
policy ⇡
⇤ and ⇡ depends on how ine�cient a level that is avoided by ⇡
⇤ is.
We next illustrate how the virtual system adapts to changes in arrival rates. We consider a
system in setting 2 where the arrival rate is 280, and we change the arrival rate to 360 at time 600.
Therefore, until time 600, levels 2 and 4 are basic and after time 600, levels 4 and 6 are basic. We
simulate the system until time 650 and we plot the average number of agents in levels 2 and 6 in
each 5 time unit interval in Figure 2. As can be seen in Figure 1, there is a quick change in each
level at time 600 and they reach levels close to the “steady state” quantities as anticipated by LP
(10).
6.1. Sta�ng levels
We next test the quality of the sta�ng levels found by the sta�ng LP in simulation experiments. We
consider two service level targets pAb = 0.10 and 0.14 with three di↵erent arrival rates presented in
Table 4. We use the sta�ng LP (31) to compute the required sta�ng levels with these parameters.
To assess the quality of our proposed solution, we also find e↵ective sta�ng levels using a
numerical procedure along with simulations. Specifically, we use a simple search algorithm where
we start with a small value for the sta�ng level and simulate the system under the proposed
policy to estimate the performance. We keep on increasing the sta�ng level until the service level
24
Setting Policy Z0 Z1 Z2 Z3 Z4 Z5 Z6 P
ab
4 ⇡
⇤ 0 0.002 0.061 1.529 11.683 2.11 9.615 13.30%4 ⇡
⇤ 0 0.003 0.12 1.171 10.316 3.978 9.412 13.49%4 ⇡ 0 0 0.02 0.49 3.939 11.796 8.755 14.34%
5 ⇡
⇤ 0 0 0.03 1.751 27.251 3.29 17.678 13.14 %5 ⇡
⇤ 0 0.001 0.054 1.462 25.12 5.829 17.534 13.27%5 ⇡ 0 0 0.003 0.206 4.6 28.258 16.932 14.53%
6 ⇡
⇤ 0 0 0.005 1.745 159.571 8.278 80.401 12.94%6 ⇡
⇤ 0 0 0.005 1.681 157.447 10.441 80.426 12.97%6 ⇡ 0 0 0 0.009 2.364 163.093 84.534 14.77%
Table 3 Results of simulation experiments: Settings 4–6
400 500 600 700 800
05
1015
Arrival Rate
Perc
enta
ge D
iffer
ence
(%)
Figure 1 Percentage di↵erence in abandonment probabilities between ⇡⇤and ⇡
requirement is satisfied and stop at the smallest feasible sta�ng level. In Table 4, we present the
solutions from the numerical search algorithm under column N and the solutions of the sta�ng
LP (31) under column N
⇤. Obviously the sta�ng LP’s solutions are very close to those found by
the numerical method: they are o↵ by at most 1 agents. We emphasize the fact that the proposed
sta�ng solutions is very accurate even in small systems where the optimal sta�ng level is as low
as 7 agents.
25
Time
Num
ber o
f Age
nts
550 575 600 625 650
010
2030
40 Level 2Level 6
Figure 2 Number of agents in levels 2 and 6 in the virtual system
p
Ab
� N N
⇤
0.10 50 11 10100 21 20250 51 50
0.14 50 7 7100 14 13250 34 33
Table 4 Comparison of sta�ng solutions
7. Conclusions and directions for further research
In this paper we considered the sta�ng and routing in CSC systems. The fact that agents can serve
multiple customers simultaneously makes the analysis of these systems more complicated than that
of traditional service systems. We first propose a routing LP to identify e↵ective routing policies.
Initially we propose a policy for the case when the arrival rate is observable. We also consider the
case when the arrival rate is unobservable and we propose a policy based on a virtual system that
automatically and seamlessly determines basic levels. We prove that these policies asymptotically
minimize the abandonment probability in steady state in large systems. We also present a sta�ng
LP and prove that the sta�ng solution found by the sta�ng LP is asymptotically optimal. Our
26
numerical experiments suggest that the proposed solution procedure and the routing policies work
remarkably well even with systems that have less than 10 agents.
There are several interesting future research directions. First, we only focus on the case when the
service requirements of the customers are exponentially distributed. When the service requirements
have a general distribution, measure-valued fluid limits (see Whitt (2006)) can be used to study
these systems. It could also be possible to establish the distribution of time in CSC systems, instead
of just the expected value, using such fluid limits. In addition, especially when �⇡ d
i
N , for some i,
the di↵usion limits may be more suitable to determine e↵ective sta�ng levels. For the overloaded
case, the analysis is similar to the (traditional) case when agents only help one customer at a
time. Finally, agents usually have di↵erent capabilities and customers have di↵erent needs in most
chat systems. How to route arriving customers in di↵erent classes to agents with di↵erent skills
e↵ectively needs to be studied as well. It may be possible to cross-train agents to provide service
in multiple service channels, such as email and chat. In such a setting it is interesting to study the
sta�ng and the routing problems by considering the di↵erent service level measures in di↵erent
channels.
References
Aksin, Z., M. Armony, V. Mehrotra. 2007. The modern call center: A multi-disciplinary perspective on
operations management research. Production and Operations Management 16 655–688.
Atar, R., C.. Giat, N. Shimkin. 2010. The cµ/✓ rule for many-server queues with abandonment. Operations
Research 58 1427–1439.
Atar, R., C. Giat, N. Shimkin. 2011. On the asymptotic optimality of the cµ/✓ rule under ergodic cost.
Queueing Systems: Theory and Applications 67(2) 127–144.
Bannan, K. 2000. Chatting up a sale. The Wall Street Journal.
Bassamboo, A., R. S. Randhawa. 2010. On the accuracy of fluid models for capacity sizing in queueing
systems with impatient customers. Operations Research 58(5) 1398–1413.
Bassamboo, A., R.S. Randhawa, A. Zeevi. 2010. Capacity sizing under parameter uncertainty: Safety sta�ng
principles revisited. Management Science 56 1668–1686.
Batt, Bob, Christian Terwiesch. 2013. The behavior of doctors under load: An empirical analysis of reputation
in online service market places. Tech. rep., The Wharton School, University of Pennsylvania.
Behzad, B., T. Tezcan. 2012. Robust design and control of call centers with flexible IVR systems. Manufac-
turing and Service Operations Management 14(3) 386–401.
Billingsley, P. 1968. Convergence of probability measures . Wiley, New York.
Broughton, K. 2001. Our experiment in online, real-time reference. Computers in Libraries 21 26–31.
27
Dai, J. G., Tolga Tezcan. 2011. State space collapse in many-server di↵usion limits of parallel server systems.
Math. Oper. Res. 36(2) 271–320.
Francoeur, S. 2001. An analytical survey of chat reference services. Reference Services Review 29 189–204.
Gamarnik, David, Assaf Zeevi. 2006. Validity of heavy tra�c steady-state approximation in generalized
Jackson networks. Ann. Appl. Probab. 16(1) 56–90.
Gans, N., G. Koole, A. Mandelbaum. 2003. Telephone call centers: Tutorial, review and research prospects.
Manufacturing and Service Operations Management 5 79–141.
Gladstones, William H, Michael A Regan, Robert B Lee. 1989. Division of attention: The single-channel
hypothesis revisited. The Quarterly Journal of Experimental Psychology 41(1) 1–17.
Harrison, J. Michael. 2000. Brownian models of open processing networks: canonical representation of
workload. Ann. Appl. Probab. 10(1) 75–103.
Hasija, Sameer, Edieal Pinker, Robert A. Shumsky. 2010. Om practice—work expands to fill the time
available: Capacity estimation and sta�ng under parkinson’s law. Manufacturing & Service Operations
Management 12(1) 1–18. doi:10.1287/msom.1080.0250. URL http://dx.doi.org/10.1287/msom.
1080.0250.
International, TELUS. 2011. Best practices: Online chat sales. Benchmarking study — White paper.
Karlin, Samuel, Howard M. Taylor. 1975. A first course in stochastic processes . 2nd ed. Academic Press
Inc., New York.
Kc, Diwas S, Christian Terwiesch. 2009. Impact of workload on service time and patient safety: An econo-
metric analysis of hospital operations. Management Science 55(9) 1486–1498.
Kleinrock, Leonard. 1976. Queueing Systems. Volume 2: Computer Applications. Wiley-Interscience, New
York. Computer Applications, Volume II.
Koyama, J.Y. 1998. http://digiref.scenarios.issues. Reference and User Services Quarterly 38 51–53.
Luo, Jun, Jiheng Zhang. 2012. Sta�ng and control of instant messaging based customer service centers.
Tech. rep., Hong Kong University of Science and Technology.
Meyn, Sean, Richard L. Tweedie. 2009. Markov chains and stochastic stability . 2nd ed. Cambridge University
Press, Cambridge.
Oder, N. 2001. The shape of e-reference. Library Journal 126 46–60.
Pashler, Harold, et al. 1994. Dual-task interference in simple tasks: Data and theory. Psychological bulletin
116 220–220.
Shae, Z., D. Garg, R Bhose, R. Mukherjee, S. Guven. 2007. E�cient internet chat services for help desk
agents. IEEE International Conference on Services Computing . IEEE Computer Society, Los Alamitos,
CA, USA, 589–596.
28
Staats, Bradley R, Francesca Gino. 2012. Specialization and variety in repetitive tasks: Evidence from a
japanese bank. Management Science 58(6) 1141–1159.
Tan, Tom, Serguei Netessine. 2012. When does the devil make work? an empirical study of the impact of
workload on worker productivity. Tech. rep., INSEAD Working Paper.
Tezcan, T., J. Zhang. 2013. Sta�ng and routing in chat service systems with general service and abandonment
times. Working paper .
Whitt, W. 2006. Fluid models for multiserver queues with abandonments. Operations Research 54 37–54.
Williams, Ruth J. 1998. Di↵usion approximations for open multiclass queueing networks: su�cient conditions
involving state space collapse. Queueing Systems: Theory and Applications 30(1-2) 27–88.
Zhang, J., J. G. Dai, B. Zwart. 2009. Law of large number limits of limited processor-sharing queues. Math.
Oper. Res. 34(4) 937–970. doi:10.1287/moor.1090.0412. URL http://dx.doi.org/10.1287/moor.
1090.0412.
29
Appendix
A. Proof of Lemma 1
We first state some preliminary results. We note that
P
Ab
i
d
i
= i⌫ (41)
hence P
Ab
i
d
i
is increasing in i. Also by (3)
P
Ab
j
0 >P
Ab
j
for j0 > j. (42)
The proof of the following result is elementary hence skipped.
Lemma 3. If for a level j, 1 < j < I, dj
= d
j
0 for some j
0< j then level j cannot be e�cient.
Also, for any e�cient level ij
⇣
1�P
Ab
ij
⌘
d
ij�
�
1�P
Ab
i
�
d
i
, (43)
for all i i
j
.
Remark 1 (Equivalent definitions). It can easily be checked that if a level i is ine�cient
than the following hold for some i1 i i2
⇣
P
Ab
i2d
i2 �P
Ab
i1d
i1
⌘
d
i
⇣
P
Ab
i2d
i2 �P
Ab
i
d
i
⌘
d
i1 +⇣
P
Ab
i
d
i
�P
Ab
i1d
i1
⌘
d
i2 , (44)
⇣
d
i2 � d
i1
⌘⇣
P
Ab
i2d
i2 �P
Ab
i
d
i
⌘
⇣
d
i2 � d
i
⌘⇣
P
Ab
i2d
i2 �P
Ab
i1d
i1
⌘
, (45)
and⇣
d
i2 � d
i1
⌘
P
Ab
i
d
i
�
⇣
d
i
� d
i1
⌘
P
Ab
i2d
i2 +⇣
d
i2 � d
i
⌘
P
Ab
i1d
i1 . (46)
Proof of Lemma 1: The routing LP (10) can be written as
minI
X
i=1
P
Ab
i
�
i
+�
I+1 (47)
st.
I
X
i=1
�
i
d
i
N, (48)
I
X
i=1
�
i
+�
I+1 � �, (49)
�
i
� 0, i= 1,2, . . . , I, I +1. (50)
30
Let y
N
, y�
and y
i
denote the dual variables associated with the constraints (48), (49) and (50),
respectively. We can write the dual of the routing LP, referred to as LP-D, as
max�y�
�Ny
N
st.
y
�
�
1
d
i
y
N
+ y
i
P
Ab
i
, i= 1,2, . . . , I
y
�
+ y
I+1 1, i= 1,2, . . . , I
y
�
� 0,
y
N
� 0,
y
i
� 0, i= 1,2, . . . , I +1.
LP-D is equivalent to the following LP, referred to as LP-DE,
max�
N
y
�
� y
N
st.
d
i
y
�
� y
N
P
Ab
i
d
i
, i= 1,2, . . . , I, (51)
y
�
1, (52)
y
�
� 0, (53)
y
N
� 0. (54)
For the rest of the proof we focus on establishing an optimal solution of LP-DE. Because an optimal
solution of LP-D can easily be derived from an optimal solution of LP-DE, we also refer to such a
solution as an optimal solution of LP-D, with a slight abuse of terminology.
We divide the proof into a few cases depending on the value of the parameters N and �. First,
assume that �
N
< d1. Consider the solution (y⇤�
, y
⇤N
) = (PAb
1 ,0) with objective value �
N
P
Ab
1 for LP-
DE. We claim that it is an optimal solution of LP-DE. Note first that it is feasible by (42). In
addition, by (51) for i= 1, for any feasible solution (y�
, y
N
) the objective function is bounded by
�
N
✓
P
Ab
1 +y
N
d1
◆
� y
N
�
N
P
Ab
1 ,
where the last inequality follows from the fact that �
N
< d1. Thus, (y⇤�
, y
⇤N
) = (PAb
1 ,0) is an optimal
solution of LP-DE and LP-D. Note also that the feasible solution given by �1 = �, and �
i
= 0,
i = 2, . . . , I for the routing LP (47) attains the same objective value with the optimal objective
function value of LP-D, therefore it must be an optimal solution of (10), proving (15).
31
Next, assume that �
N
= d
i
⇤ for an e�cient level i⇤. By (44) and (45) and the fact that i
⇤ is
e�cient for any i1 < i
⇤< i2 and i1, i2 2N
c
P
Ab
i
⇤ d
i
⇤�P
Ab
i1d
i1
d
i
⇤� d
i1
P
Ab
i2d
i2 �P
Ab
i1d
i1
d
i2 � d
i1
P
Ab
i2d
i2 �P
Ab
i
⇤ d
i
⇤
d
i2 � d
i
⇤, (55)
where by convention ·/0 =1 and d
i2 � d
i
⇤> d
i1 by Lemma 3 and the fact that i1, i2 2N
c. Let y⇤�
be such that
P
Ab
i
⇤ d
i
⇤�P
Ab
i1d
i1
d
i
⇤� d
i1
y
⇤�
P
Ab
i2d
i2 �P
Ab
i
⇤ d
i
⇤
d
i2 � d
i
⇤, for all i1, i2 2N
c
, i1 < i
⇤ and i2 > i
⇤. (56)
The existence of such y
⇤�
is guaranteed by the fact that i⇤ is assumed to be e�cient and by (55).
Also, set
y
⇤N
= d
i
⇤y
⇤�
�P
Ab
i
⇤ d
i
⇤. (57)
First note that by the fact that I is e�cient, dI
> d
i
⇤ and by (56) with i2 = I, y⇤�
1. Using (56)
and (57) it is easily checked that (y⇤�
, y
⇤N
) satisfies (51) for all i 2 N
c. Also, for i 2 N , (y⇤�
, y
⇤N
)
satisfies (51) by (41), the definition of N and the fact that (y⇤�
, y
⇤N
) satisfies (51) for i2N
c. Hence,
(y⇤�
, y
⇤N
) is a feasible solution of LP-DE with the objective function value PAb
i
⇤ d
i
⇤ . The optimality of
(y⇤�
, y
⇤N
) for LP-DE (and so for LP-D) follows from the fact that for any feasible solution (y�
, y
N
)
the objective function satisfies
�
N
y
�
� y
N
= d
i
⇤y
�
� y
N
P
Ab
i
⇤ d
i
⇤
by our assumption that �
N
= d
i
⇤ . Note also that the feasible solution given by �
i
⇤ = �, �i
= 0 for
i 6= i
⇤ for the routing LP (47) attains the same objective value as the optimal objective function
value of LP-D, therefore it must be an optimal solution of (10), proving (16) in this case.
Now assume that dI
N > �> d1N and that �
N
6= d
i
⇤ for any i
⇤2F . Let i⇤
j+1 be defined as in (14)
and define
(y⇤�
, y
⇤N
) =
0
@
P
Ab
i
⇤j+1
d
i
⇤j+1
�P
Ab
i
⇤jd
i
⇤j
d
i
⇤j+1
� d
i
⇤j
,
P
Ab
i
⇤j+1
d
i
⇤j+1
d
i
⇤j�P
Ab
i
⇤jd
i
⇤jd
i
⇤j+1
d
i
⇤j+1
� d
i
⇤j
1
A
. (58)
Also define
�=d
i
⇤j+1
�
�
N
d
i
⇤j+1
� d
i
⇤j
.
Using (51) for i⇤j
and i
⇤j+1, we have for any feasible solution (y
�
, y
N
)
�
N
y
�
� y
N
= �(di
⇤jy
�
� y
N
)+ (1� �)(di
⇤j+1
y
�
� y
N
)d
i
⇤j+1
�
�
N
d
i
⇤j+1
� d
i
⇤j
P
Ab
i
⇤jd
i
⇤j+
�
N
� d
i
⇤j
d
i
⇤j+1
� d
i
⇤j
P
Ab
i
⇤j+1
d
i
⇤j+1
.
32
On the other hand, it is easy to verify that (y⇤�
, y
⇤N
) defined in (58) achieves this best possible value.
Therefore it is enough to show that (y⇤�
, y
⇤N
) is a feasible solution for LP-DE to conclude that it is
an optimal solution for LP-DE.
By the monotonicity property (41) and Lemma 3, (y⇤�
, y
⇤N
) satisfies the constraints (52), (53) and
(54). Also, it satisfies (51) for i = i
⇤j
and i = i
⇤j+1. In fact, these two constraints are tight at this
point. We next prove that (y⇤�
, y
⇤N
) satisfies (51) for i 6= i
⇤j
, i
⇤j+1. For i < i
⇤j
,
d
i
y
⇤�
� y
⇤N
= d
i
⇤j+1
y
⇤�
� y
⇤N
� (di
⇤j+1
� d
i
)y⇤�
= P
Ab
i
⇤j+1
d
i
⇤j+1
� (di
⇤j+1
� d
i
)
0
@
P
Ab
i
⇤j+1
d
i
⇤j+1
�P
Ab
i
⇤jd
i
⇤j
d
i
⇤j+1
� d
i
⇤j
1
A
. (59)
Since level i⇤j
is e�cient, by (45) and Lemma 3
P
Ab
i
⇤j+1
d
i
⇤j+1
�P
Ab
i
⇤jd
i
⇤j
d
i
⇤j+1
� d
i
⇤j
�
P
Ab
i
⇤j+1
d
i
⇤j+1
�P
Ab
i
d
i
d
i
⇤j+1
� d
i
.
This implies by (59) that d
i
y
⇤�
� y
⇤N
P
Ab
i
d
i
. So (51) is satisfied for any i < i
⇤j
by (y⇤�
, y
⇤N
). For
i > i
⇤j+1, a similar argument applies by using the e�ciency of level i⇤
j+1. Specifically, by (45)
(di
� d
i
⇤j)PAb
i
⇤j+1
d
i
⇤j+1
(di
⇤j+1
� d
i
⇤j)PAb
i
d
i
+(di
� d
i
⇤j+1
)PAb
i
⇤jd
i
⇤j. (60)
Then
d
i
y
⇤�
� y
⇤N
= d
i
⇤jy
⇤�
� y
⇤N
+(di
� d
i
⇤j)y⇤
�
= P
Ab
i
⇤jd
i
⇤j+(d
i
� d
i
⇤j)
0
@
P
Ab
i
⇤j+1
d
i
⇤j+1
�P
Ab
i
⇤jd
i
⇤j
d
i
⇤j+1
� d
i
⇤j
1
A
P
Ab
i
d
i
,
where the last inequality follows from (60).
If Ui
⇤j= ;, then the proof is complete. Also, for all i 2 N , (y⇤
�
, y
⇤N
) satisfies (51), as described
above if (y⇤�
, y
⇤N
) satisfies (51) for all i2N
c. So next we consider Uij\N
c. For any i2 U
i
⇤j\N
c, let
�
j+1 =P
Ab
i
⇤j+1
d
i
⇤j+1
�P
Ab
i
d
i
P
Ab
i
⇤j+1
d
i
⇤j+1
�P
Ab
i
⇤jd
i
⇤j
and �
j
= 1� �
j+1. By (41), �j
, �
j+1 2 [0,1]. It follows from Lemma 4 below and (46) that
d
i
�
j+1di⇤j+ �
j
d
i
⇤j+1
. (61)
Then
d
i
y
⇤�
� y
⇤N
�
j+1
⇣
d
i
⇤jy
⇤�
� y
⇤N
⌘
+ �
j
⇣
d
i
⇤j+1
y
⇤�
� y
⇤N
⌘
33
�
j+1PAb
i
⇤jd
i
⇤j+ �
j
P
Ab
i
⇤j+1
d
i
⇤j+1
= P
Ab
i
d
i
,
where the first inequality follows from (61) and the second inequality follows from the fact that
(y⇤�
, y
⇤N
) satisfies the constraint (51) for i⇤j
and i
⇤j+1. So (y⇤
�
, y
⇤N
) satisfies the constraint (51) for all
1 i I. Therefore, (y⇤�
, y
⇤N
) is an optimal solution for LP-DE and LP-D. In addition, the feasible
solution given by (16) for the routing LP (47) attains the same objective value as the optimal
objective value of LP-D, therefore it must be an optimal solution of (10), proving (16) in this case.
Finally, we study the case where �� d
I
N . Consider the solution (y⇤�
, y
⇤N
) = (1, dI
(1�P
Ab
I
)) with
objective value �
N
� d
I
(1� P
Ab
I
) for LP-DE. We claim that it is an optimal solution of LP-DE.
Note first that it is feasible by (6). In addition, by (51) for i= I and (52), for any feasible solution
(y�
, y
N
) the objective function is bounded by �
N
� d
I
(1�P
Ab
I
). Thus, (y⇤�
, y
⇤N
) is an optimal solution
of LP-DE and LP-D. Note also that the feasible solution given by �
I
= d
I
N , �I+1 = �� d
I
N , and
�
i
= 0, i= 1, . . . , I � 1 for the routing LP (47) attains the same objective value with the optimal
objective function value of LP-D, therefore it must be an optimal solution of (10), proving (17).
⇤
A.1. Proof of (61)
We now focus on the ine�cient levels in U
i
⇤j\ N
c (when U
ij6= ;) for an e�cient level i
j
. For
notational simplicity we assume N = ; and use only the notation U
i
⇤jthe rest of this section. (We
do not make use of this assumption in the proof.) For an ine�cient level i, we denote the set
of all pairs (i1, i2) with i1 < i < i2 that satisfy (45) by J (i). For any i 2 U
ij, by the definition of
ine�ciency, there exists a pair (k, j)2J (i). However, the definition itself does not require (ij
, i
j+1)
to be one of the pairs in J (i). In this section we prove that (ij
, i
j+1)2J (i).
Let a
i
= P
Ab
i
d
i
and consider the set of pairs {(di
, a
i
) : i = {1,2, . . . , I}} on a two-dimensional
plane. Assume that dk
6= d
j
and let Lk,j :R!R denote the (straight) line passing through points
(dk
, a
k
) and (dj
, a
j
), hence,
L
k,j(x) =a
j
� a
k
d
j
� d
k
x�
a
j
� a
k
d
j
� d
k
d
k
+ a
k
, x2R.
From here on when we consider Lk,j we assume without further mention that dk
6= d
j
. For k < i < j,
we have
L
k,j(di
) = a
j
d
i
� d
k
d
j
� d
k
+ a
k
d
j
� d
i
d
j
� d
k
.
34
If a level i is ine�cient then by (46) either dk
= d
i
for some k < i or
a
i
�L
k,j(di
) (62)
for some (k, j)2 J(i).
Lemma 4. Let ij
2F . For any i2 U
ij(61) holds.
Note that (61) is equivalent to
a
i
�L
ij ,ij+1(di
). (63)
We use the following results in the proof that are immediate from algebraic manipulations. Let
k < i and k < j and assume that Lk,i(x)L
k,j(x) for some x> d
k
, then
L
k,i(x)L
k,j(x) for all x> d
k
(64)
Similarly, if for k < i and j < i L
k,i(x)L
j,i(x) for some x< d
i
, then
L
k,i(x)L
j,i(x) for all x< d
i
. (65)
Let i < j and k < l and assume that there exists x1 >x0 such that Li,j(x)L
k,l(x) for some x< x0,
and L
i,j(x)�L
k,l(x) for some x> x1, then
L
i,j(x)L
k,l(x) for all x< x0, and L
i,j(x)�L
k,l(x) for all x> x1. (66)
Proof of Lemma 4: We prove the result using an induction argument. Because level i is ine�-
cient, by Lemma 5 below, (k1, ij+1)2J (i) for some k1 < i. Hence, by (62) (note that by Lemma 3
and the fact that ij+1 is e�cient we have d
i
< d
ij+1for all i < i
j+1)
a
i
�L
k1,ij+1(di
). (67)
Now we proceed by induction. Let i = i
j
+ 1. Assume that k1 < i
j
as otherwise the proof is
complete. Because level ij
is e�cient
a
ij<L
k1,ij+1(dij). (68)
Since a
ij= L
ij ,ij+1(dij), we have L
ij ,ij+1(x) L
k1,ij+1(x) for all x d
ij+1by (65). The result then
follows from (67).
Now assume that for i2 U
ij, i > i
j
+1 and
a
i
0�L
ij ,ij+1(di
0)
35
for all ij
+1 i
0 i� 1. By Lemma 5, there exists 1 k1 < i such that (k1, ij+1)2J (i). If k1 i
j
,
the result follows as above using (68). Now assume that k1 > i
j
. By the induction argument
a
k1 =L
k1,ij+1(dk1)�L
ij ,ij+1(dk1).
Therefore, by (65)
L
k1,ij+1(x)�L
ij ,ij+1(x)
for x< d
ij+1. Then (63) follows from (67). ⇤
Lemma 5. Let ij
2F . For any i2 U
ij,
a
i
�L
k,ij+1(di
) (69)
for some 1 k < i.
Proof of Lemma 5: Assume that level i is ine�cient. Then by (46) either di
= d
j
for some j < i
or
a
i
�L
j1,j2(di
). (70)
for some j1 < i< j2.
First assume that di
= d
i
0 for some i
0< i. Then, by Lemma 3 d
ij+1> d
i
= d
i
0 . Hence, we have
a
i
�L
i
0,ij+1(d
i
)
by (64) completing the proof in this case.
Now assume that di
> d
i�1 and that (70) holds. It follows from (64) and Lemma 3 that dj2 > d
i
.
We prove the result in this case recursively by induction. First suppose i= i
j+1� 1. In this case, if
j2 = i
j+1 the result follows. Otherwise, we must have j2 > i
j+1. The fact that level ij+1 is e�cient
implies
L
j1,ij+1(dij+1
) = a
ij+1<L
j1,j2(dij+1
),
hence by (64)
L
j1,ij+1(x)L
j1,j2(x), for all x> d
j1 . (71)
The result (69) with k= j1 follows from (70) and (71) (with x= d
i
since d
i
> d
j1).
36
Now suppose i < i
j+1 � 1 and by induction that for all i0 with i < i
0< i
j+1
a
i
0�L
k1(i0),ij+1(d
i
0) (72)
for some k1(i0)< i
0. By the discussion above, if di
= d
i�1 the proof is immediate, therefore we focus
on the case when d
i
> d
i�1. If di > d
i�1 then it follows from (64) and Lemma 3 that dj2 > d
i
. Also,
by Lemma 3 and the fact that ij+1 is e�cient we have d
j2 < d
ij+1. Hence for the rest of the proof
the following holds
d
i�1 < d
i
< d
j2 < d
ij+1. (73)
If j2 � i
j+1, then the result (69) (with k = j1) follows in a similar way to the discussion leading
to (71). So we now focus on the case where j2 < i
j+1. We can assume the pair (j1, j2) we pick is the
one such that j2 =min{j : (j1, j)2J (i)}. By the induction assumption (72), for this j2
a
j2 �L
k1(j2),ij+1(dj2). (74)
There are three di↵erent cases to be analyzed separately.
(i) k1(j2)< j1 < i< j2
(ii) j1 k1(j2)< i< j2
(iii) j1 < i k1(j2)< j2
Case (i): If ai
�L
k1(j2),ij+1(di
), then result (69) with k= k1(j2) follows immediately. So we focus
on the case where
a
i
<L
k1(j2),ij+1(di
). (75)
We show below that in this case,
a
j1 L
k1(j2),ij+1(dj1). (76)
We now show the result (69) holds with k = j1 using (75) and (76). Figure 3 gives a graphic
demonstration of the argument. Let Lj1,j2 denote the line that passes through the points (dj1 , aj1)
and (dj2 ,L
k1(j2),ij+1(dj2)). Since (74) holds, by (64), we have that
L
j1,j2(x)� L
j1,j2(x), for all x� d
j1 . (77)
On the other hand, since (76) holds, by (65), we have that
L
k1(j2),ij+1(x)�L
j1,ij+1(x), for all x d
ij+1.
37
Plugging x= d
j2 in the above yields Lk1(j2),ij+1(dj2) = L
j1,j2(dj2)�L
j1,ij+1(dj2). This, again by (64),
implies that
L
j1,j2(x)�L
j1,ij+1(x), for all x� d
j1 . (78)
The result follows from (70), (77) and (78) (with x= d
i
since d
i
� d
j1).
To complete the proof in Case (i), we next prove (76). We first show that (74) and the opposite
of (76) implies that
L
j1,j2(x)�L
k1(j2),ij+1(x), for all dj1 x d
j2 . (79)
(In other words, the line segment connecting (dj1 , aj1) and (d
j2 , aj2) lies above the line passing
through (dk1(j2), ak1(j2)) and (d
ij+1, a
ij+1).) Let Lj1,j2 be defined as above. By (74) and (64)
L
j1,j2(x)L
j1,j2(x), for all x� d
j1 (80)
and if (76) does not hold then by (65)
L
j1,j2(x)�L
k1(j2),ij+1(x), for all x d
j2 (81)
We have (79) by (80) and (81). By (79) and (70), we must have that a
i
� L
k1(j2),ij+1(di
), contra-
dicting (75), hence (76) holds.
a
k1(j2)
d
k1(j2)
a
ij+1
d
ij+1
a
j1
d
j1
a
j2
d
j2
a
i
d
i
Figure 3 Graphic Presentation in Case (i).
Case (ii): If ak1(j2) <L
j1,j2(dk1(j2)), then this and (74) implies that
L
j1,j2(x)�L
k1(j2),ij+1(x), for all dk1(j2) x d
j2 . (82)
38
Since d
k1(j2) d
i
d
j2 , the result (69) with k = k1(j2) follows immediately from (70) and from
plugging x= d
i
in (82).
Now assume that ak1(j2) �L
j1,j2(dk1(j2)). This implies that
L
k1(j2),ij+1(dk1(j2))�L
j1,j2(dk1(j2)).
It follows from (74) that
L
k1(j2),ij+1(dj2)L
j1,j2(dj2).
By (66),
L
k1(j2),ij+1(x)�L
j1,j2(x), for all x d
k1(j2).
In particular, the above holds for x= d
j1 since d
j1 d
k1(j2). Thus, by (65),
L
k1(j2),ij+1(x)�L
j1,ij+1(x), for all x d
ij+1.
Since d
j2 d
ij+1, the above inequality holds for x= d
j2 , this together with (74) implies that aj2 �
L
j1,ij+1(dj2). By (64),
L
j1,j2(x)�L
j1,ij+1(x), for all x� d
j1 . (83)
The result (69) with k= j1 follows from (70) and (83) (with x= d
i
since d
i
� d
j1).
Case (iii): Since j2 is the smallest level such that (j1, j2)2J (i) we must have
a
i
L
j1,k1(j2)(di
). (84)
Also note that d
ij+1> d
k1(j2) � d
i
> d
j1 by (73). (84) together with (70) implies L
j1,k1(j2)(di
) �
L
j1,j2(di
). Since d
k1(j2) � d
i
by (64),
a
k1(j2) =L
k1(j2),ij+1(dk1(j2)) =L
j1,k1(j2)(dk1(j2))�L
j1,j2(dk1(j2)).
On the other hand, (74) implies that
L
k1(j2),ij+1(dj2)L
j1,j2(dj2).
By (66), we have
L
k1(j2),ij+1(x)L
j1,j2(x) for all x� d
j2 .
In particular, the above inequality holds for x= d
ij+1since d
ij+1� d
j2 . Thus, by (64), (83) holds
in this case as well. The result (69) with k = j1 follows from (70) and (83) (with x = d
i
since
d
i
� d
j1). ⇤
39
B. Queueing equations
In this section we provide the details of the queueing equations for the CSC systems and set the
notation for the rest of results in the appendix.
B.1. Notation
All random variables and processes are defined on a common probability space (⌦,G, P ) unless
specified otherwise. The symbols N,R and R+ are used to denote nonnegative integers, real numbers
and nonnegative real numbers, respectively. For d 2 N, Rd denotes the d-dimensional Euclidean
space; thus, R= R1. The space of functions f : R+ ! Rd that are right-continuous on [0,1) and
have left limits in (0,1) is denoted by D(R+,Rd) or simply Dd ; similarly, with T > 0, the space of
functions f : [0, T ]!Rd that are right-continuous on [0, T ) and have left limits in (0, T ] is denoted
by D([0, T ],Rd). For f 2 Dd, f(t�) denotes its left limit at t > 0. Each stochastic process whose
sample paths are in Dd is considered to be a Dd-valued random element. The space Dd is assumed
to be endowed with the u.o.c. topology (see Billingsley (1968)). For a function f :R!Rd with d
being some positive integer, we say that t is a regular point of f if f is di↵erentiable at t and use
f(t) to denote its derivative at t. We use E
�
[·] to denote the conditional expectation and P
�
{·} to
denote the conditional probability given that (Q(0),Z(0)) is distributed according to �. Similarly,
we use E
x
[·] to denote the conditional expectation and P
x
{·} to denote the conditional probability
given that (Q(0),Z(0)) = x.
B.2. Queueing Equations
In this section we introduce the queueing equations for the CSC systems. Fix � and let Z�
i
(t) denote
the number of agents serving i customers and Q
�(t) denote the number of customers in queue at
time t. Also, let {Si
: i= 1, . . . , I+1} denote a set of independent rate 1 Poisson processes and A
�(t)
denote the number of arrivals at the system by time t which is also assumed to be independent
of {Si
: i= 1, . . . , I +1}. We denote by A
�
i
(t) the number of customers who are routed to an agent
serving i customers for i= 0,1, . . . , I � 1 and by A
�
I
(t) the number of customers who are routed to
the queue upon arrival by time t in the �th system. We set Z� = (Z�
i
(0); i= 0,1, . . . , I, t� 0) and
A
� = (A�(t),A�
i
(t); i= 0,1, . . . , I, t� 0). Let
D
�
i
(t) = S
i
✓
d
i
Z
t
0
Z
�
i
(s)ds
◆
, i= 1, . . . , I
and
D
�
I+1(t) = S
I+1
✓
�
Z
t
0
Q
�(s)ds
◆
40
denote the number of customers who leave the system while receiving service from an agent at
level i and the number of abandonments from queue by time t, respectively. The following queueing
equations are satisfied under any policy for t� 0.
Z
�
0 (t) =Z
�
0 (0)�A
�
0(t)+D
�
1 (t) , (85)
Z
�
i
(t) =Z
�
i
(0)+A
�
i�1(t)�A
�
i
(t)+D
�
i+1 (t)�D
�
i
(t) , i= 1, . . . , I � 2, (86)
Z
�
I�1(t) =Z
�
I�1(0)+A
�
I�2(t)�A
�
I�1(t)+L
�
q
(t)�D
�
I�1 (t) , (87)
Q
�(t) =Q
�(0)+A
�
I
(t)�D
�
I
(t)�D
�
I+1 (t)+L
�
q
(t), (88)I
X
i=0
A
�
i
(t) =A
�(t), (89)
Z
�
i
(t)� 0,A�
i
(0) = 0 and A
�
i
is nondecreasing, i= 0,1, . . . , I. (90)I
X
i=0
Z
�
i
(t) =N
�
, (91)
where
L
�
q
(t) =
Z
t
0
1�
Q
�(s) = 0
dD
�
I
(s).
By (91), Z�
I
(t) =N
�
�
P
I�1i=0 Z
�
i
(t), for all t� 0, hence
D
�
I
(t) = S
I
d
I
N
�
t�
Z
t
0
I�1X
i=0
Z
�
i
(s)ds
!!
.
Also, an arrival can be routed to an agent at level i only when there is at least one agent at that
level, hence,
Z
t
0
1�
Z
�
i
(s�) = 0
dA
�
i
(s) = 0, t� 0.
In addition, because we focus on non-idling policies, customers wait in the queue only when all
agents are at level I, therefore
Q
�(t)
I�1X
i=0
Z
�
i
(t)
!
= 0, t� 0. (92)
Additional equations under the proposed policy: Note that the policy proposed in §4.1 is a static
priority policy once the basic levels i
�
j
and i
�
j+1 are fixed. For levels below i
�
j
, those with lower
indexes have higher priorities. Levels in U
�
ijhave lower priorities than those levels below i
�
j
but
higher priorities than i
�
j
. Also their priorities among those levels in this set are set in the reverse
order of their indexes. All levels above i
�
j+1 (including i
�
j+1) have lower priorities than levels below
i
�
j+1. Also for two levels above i
�
j+1, levels in N
0 have lower priorities than those levels that are not
in N
0. Levels that are not in N
0 have decreasing priority in their indexes and levels that are in
41
N
0 have increasing in their indexes. We can re-index the levels using {v
�
i
} so that v�i
denotes the
priority index of level i and v
�
i
< v
�
i
0 if level i has priority over i0. We note that under the proposed
policy, for 0< t1 < t2, we have
A
�
i
(t2)�A
�
i
(t1) = 0, ifX
{i0:v�i0<v
�i }
Z
�
i
0(s)> 0, for all s2 [t1, t2], i= 1, . . . , I � 1. (93)
C. Proof of Theorem 1
Fix � and let ⇡
� denote a non-idling policy. We first prove that under ⇡
� there exists a unique
steady state distribution �(⇡�) for the processes (Z�
,Q
�). We prove this result using Foster’s
criteria. Choose K > 0 large enough so that �(K �N
�)+ Id
I
N
�
��>�c, for some c > 0. We say
a state (Z,Q) is feasible if (91) and (92) are satisfied. (Throughout the proof we use the notation
defined in §B.) Given a (feasible) state (Z,Q), with Z = (Z0,Z1, . . . ,ZI
), where Z
i
is the number
of agents at level i and Q is the queue length, define f(Z,Q) =P
I
i=0Zi
+Q. Then, for any state
(Z,Q) with f(Z,Q)>K, for the generator of the underlying Markov chain �, we have
�f(Z,Q) = �� d
I
N
�
� �Q<�c
because the policy is assumed to be non-idling. Therefore (Z�
,Q
�) is positive recurrent for each �
by Foster’s criteria (see for example Meyn and Tweedie (2009)).
Because under any non-idling policy there exists a unique stationary distribution, by Meyn and
Tweedie (2009) and (85)–(88), we have for each �
P
�,⇡
�(Ab) = lim
T!1
Ab
�(T )
T�
=E
�(⇡�)
h
P
I
i=1 i⌫Z�
i
(0)+ �Q
�(0)i
�
. (94)
Obviously, under �(⇡�)
I
X
i=1
Z
�
i
(0)N. (95)
Because the underlying chain is ergodic
limT!1
D
�
i
(T )
T
=E
�(⇡�)
h
d
i
Z
�
i
(0)i
a.s.,
for i= 0,1, . . . , I and
limT!1
D
�
I+1(T )
T
=E
�(⇡�)
⇥
�Q
�(0)⇤
a.s.
In addition, by stationarity
�=I
X
i=0
E
�(⇡�)
h
d
i
Z
�
i
(0)i
+E
�(⇡�)
⇥
�Q
�(0)⇤
. (96)
42
Consider the following LP
min{Q,Zi;i=0,1,...,I}
P
Ab
st. �Q+I
X
i=1
i⌫Z
i
= P
Ab
�,
I
X
i=0
Z
i
N
�
,
�Q+I
X
i=1
d
i
Z
i
� �,
Q,Z
i
� 0, i= 0,1, . . . , I.
(97)
Denote the optimal solution of this LP by P
Ab(�,N�). By (94), (95), and (96), we have that
P
�,⇡
�(Ab)� P
Ab(�,N�).
Also, LP (97) is equivalent to LP (10), which can be seen by setting �
i
= d
i
Z
i
and �
I+1 = �Q there
and using (2). Therefore,
P
�,⇡
�(Ab)� P
Ab(1,��1N
�). (98)
Then taking the limit on both sides of (98) as � goes to infinity and using the fact
lim�!1
P
Ab(1,��1N
�) = P
Ab(1,N),
which follows from (9), and the fact that the optimal solution of the LP is a continuous function
of its constraints gives the desired result. ⇤
D. Proof of Theorem 2
In this section we prove Theorem 2. (Throughout the proof we use the notation defined in §B.)
The proof is based on analyzing the (fluid) limits of the fluid scaled processes (n�1Z
n
, n
�1Q
n).
Specifically we find the steady state of the fluid limits by establishing the fluid model equations.
Then we show that the steady state of the fluid scaled processes are tight. Then we combine these
results to complete the proof. For the case when there is only one basic level in the limit the proof
is slightly di↵erent but the idea is similar.
We analyze the fluid limits of the queuing processes under the proposed policies in §D.1 and
then prove the tightness of the stationary distributions of the fluid scaled processes in §D.2. We
present the proof in §D.3
43
D.1. Analysis of the fluid model
In this section we analyze the fluid limits of the chat systems under the proposed policies. First we
establish the fluid model equations that are satisfied by the fluid limits and then find the steady
state of the fluid model solutions.
Consider the asymptotic regime where (9) holds. Consider the following fluid model equations.
Z0(t) = Z0(0)� A0(t)+ d1
Z
t
0
Z1(s)ds, (99)
Z
i
(t) = Z
i
(0)+ A
i�1(t)� A
i
(t)+ d
i+1
Z
t
0
Z
i+1(s)ds� d
i
Z
t
0
Z
i
(s)ds, i= 1, . . . , I � 2, (100)
Z
I�1(t) = Z
I�1(0)+ A
I�2(t)� A
I�1(t)+ L
q
(t)� d
I�1
Z
t
0
Z
I�1(s)ds, (101)
Q(t) = Q(0)+ A
I
(t)� d
I
Z
t
0
Z
I
(s)ds� �
Z
t
0
Q(s)ds+ L
q
(t), (102)
I
X
i=0
˙A
i
(t) = ˙A(t) = 1, (103)
Z
I
(t) =N �
I�1X
i=0
Z
i
(t), (104)
X
{k:vkvi}
˙A
k
(t) = 1 if Zi
0(t)> 0, for some i
0 with v
i
� v
i
0, i= 0,1, . . . , I � 1, (105)
˙L
q
(t) = 0 if Q(t)> 0, (106)
Q(t)I�1X
i=0
Z
i
(t) = 0, (107)
Z
i
(t)� 0, Q(t)� 0, i= 0,1, . . . , I, (108)
A
i
, L
q
are non-decreasing, i= 0,1, . . . , I, (109)
where the priorities of levels are found based on i
⇤(1,N) as explained in §B.2. We refer to
(99)–(109) as the fluid model and to (Q, Z, A, L
q
) with Z =�
Z
i
(t); i= 0,1, . . . , I, t� 0�
and A=
(A(t), Ai
(t); i= 0,1, . . . , I, t� 0) which satisfies these equations as a fluid model solution. It can be
shown as in Dai and Tezcan (2011) that every fluid model solution is Lipschitz and so di↵erentiable
a.e.
Because the proposed policy depends on the values of � and N
�, if i⇤(1,N) has only one level,
there may be a “discontinuity” in the proposed policies along the sequence of systems. Specifically,
the proposed policy ⇡
�,⇤ may be di↵erent for each � and may fluctuate between di↵erent policies as
�!1. We prove Theorem 2 separately for the cases when such discontinuity is present and when
it is not. Under the following assumption we will show that the proposed policy is independent of
� for large �.
Assumption 1. One of the following conditions holds
44
i) The set i⇤(1,N) has two elements;
ii) The set i⇤(1,N) has only one element and all the levels are e�cient;
iii) N
� = �N for some N � 0 and for � large enough.
If Assumption 1 holds, one can work with the fluid model to complete the proof of Theorem 2.
Otherwise, one needs to work with the fluid limits as described in Proposition 3. We next show
that the fluid model is obtained from the underlying queueing system equations. The fluid scaling
of the queueing processes are defined as Z
�(t) = �
�1Z
�(t), Q�(t) = �
�1Q
�(t), A�(t) = �
�1A
�(t)
and L
�
q
(t) = �
�1L
�
q
(t). Assume that
�
Z
�(0), Q�(0)�
!
�
Z(0), Q(0)�
as �!1 a.s. (110)
with Z(0) =�
Z
i
(0); i= 0,1, . . . , I�
.
Proposition 2. Consider a sequence of chat service systems and assume that (9), (110) and
Assumption 1 hold. Then the sequence {
�
Z
�
, Q
�
, A
�
, L
�
q
�
} is tight a.s. in the Skorohod space
endowed with the u.o.c. topology and every limit�
Z, Q, A, L
q
�
is a fluid model solution.
Proof of Proposition 2: Consider a sequence of chat service systems and assume that (9), (110)
and Assumption 1 hold. The proof of the tightness of the sequence {
�
Z
�
, Q
�
, A
�
, L
�
q
�
} is standard
(see Dai and Tezcan (2011)) hence we skip the details. Also, the fact that fluid limits satisfy the
fluid model equations can be proved as in there. We also note that all the fluid model equations
(99)–(109) except (105) are satisfied by the fluid limits under any sequence of non-idling policies.
We next show that the fluid limits under the proposed policy satisfy the policy specific equation
(105).
Fix a fluid limit�
Z, Q, A, L
q
�
and assume that Zi
(t)> 0 for some i I � 1. Then, there exists a
subsequence, denoted again by � and ! 2⌦ such that
�
Z
�
, Q
�
, A
�
, L
�
q
�
!
�
Z, Q, A, L
q
�
, u.o.c. as �!1. (111)
By the continuity of the fluid model solutions, (111) implies that
Z
�
i
(s)> 0 (112)
for s2 [t� �, t+ �], for some �> 0 small enough and for all � large enough. Under Assumption 1(i),
by the continuity of an LP on its constraints and because there can be at most two basic levels, for
� large enough we have i
⇤(�,N�) = i
⇤(1,N). Under Assumption 1(iii) we have i
⇤(�,N�) = i
⇤(1,N)
for all � large enough. Under Assumption 1(ii) the set of basic levels may be di↵erent for two
di↵erent �’s, however, because all the levels are e�cient the policy is a strict priority rule giving
priority to levels with a lower index for each �. Condition (105) then follows from (89), (93), (111)
and (112). ⇤
45
Steady state of the fluid limits: Next we establish the steady state of the fluid model. Fix N and
consider the optimal solution �
⇤ (1,N) = (�⇤i
(1,N) ; i= 0,1, . . . , I) of (10). Let z⇤ be defined as in
(18) and
q
⇤ =
(
0, if 1 d
I
N
1�dIN
�
, if 1> d
I
N.
(113)
Theorem 7. Let M >N be such that q⇤ <M . For any fluid model solution�
Z, Q, A, L
q
�
with�
�(Q(0), Z(0))�
�
< 2M and for any ✏> 0 there exists T (M, ✏)> 0 such that
�
�(Q(t), Z(t))� (q⇤, z⇤)�
�
< ✏ (114)
for all t� T (M, ✏).
We present the proof of this result in §H.1.
When Assumption 1 does not hold: Next we analyze the fluid limits when Assumption 1 does
not hold and prove a result similar to Theorem 7. In this case we work with the fluid limits instead
of the fluid model solutions as the fluid limits may not satisfy (105) because of the aforementioned
“discontinuity” in the proposed policy in the sequence of CSC systems.
Proposition 3. Consider a sequence of chat service systems and assume that (9) and (110)
hold. Also assume that i⇤(1,N) has only one element,
�
Z
�
, Q
�
, A
�
, L
�
q
�
!
�
Z, Q, A, L
q
�
, a.s. u.o.c. as �!1 (115)
and that
limsup�!1
P
�
Q
�(0)>M
< ✏
for some M and ✏> 0. Then, there exists T > 0 large enough such that
limsup�!1
P
�
�
�(Q�(t), Z�(t))� (q⇤, z⇤)�
�
> ✏
< 2✏ (116)
for all t� T .
We present the proof in §H.2.
D.2. Convergence of steady state quantities
Consider the asymptotic regime where (9) holds. Let �(⇡�) denote the stationary distribution of
(Z�(t), Q�(t)) under a non-idling policy ⇡
�. The existence and uniqueness of �(⇡�) follow from
the proof of Theorem 1. We next show that the sequence of fluid scaled stationary distributions is
tight.
46
Theorem 8. Consider a sequence of chat service systems that satisfies (9) under a sequence of
non-idling policies {⇡�
} and let�
Z
�(0), Q�(0)�
be distributed according to �(⇡�). Then the sequence�
(Z�(0), Q�(0))
is tight and the sequence�
Q
�(0)
is uniformly integrable.
A proof is presented in §H.3.
D.3. Proof of Theorem 2
Consider a sequence of chat service systems that satisfies (9). Let �� denote the stationary distri-
bution in the �th system. The first claim follows from the second one, Theorem 8, and the fact
that
P
�
�
�(Ab) = limT!1
Ab
�(T )
T�
=E
�
�
h
�Q
�(0)+P
I
i=1 i⌫Z�
i
(0)i
�
. (117)
We focus on proving the second part for the rest of the proof. Assume that the initial state
(Q�(0),Z�(0)) has distribution �
�. By Theorem 8, the sequence {(Q�(0), Z�(0))} is tight. Therefore,
it is su�cient to show that every convergent subsequence of (Q�(0), Z�(0)) converges to the same
limit (q⇤, z⇤). To this end consider a convergent subsequence, denoted again by {�} for notational
simplicity. We show that
(Q�(0), Z�(0))! (q⇤, z⇤) as n!1 in probability. (118)
To prove (118) it is enough to show that
limsup�!1
P
�
�
�(Q�(0), Z�(0))� (q⇤, z⇤)�
�
> ✏
< ✏
0 (119)
for any ✏, ✏
0> 0. Therefore, fix ✏> 0 and ✏
0> 0. By Theorem 8, there exists M > 0 such that
limsup�!1
P
�
�
�(Q�(0), Z�(0))�
�
>M
< ✏
0/2. (120)
Using an argument similar to that in Theorem A.1 in Dai and Tezcan (2011) and using Theorem
4.4 in Billingsley (1968), the sequence
�
(Q�(0), Z�(0), Q�
, Z
�
, A
�
, L
�
q
, S
�
(121)
is tight, where S
� = (S�
1 , S�
2 , . . . , S�
I+1) and S
�
i
(t) = �
�1S
i
(�t), i= 1,2, . . . , I +1. Consider a conver-
gent subsequence, again denoted by �, and let the limit of this subsequence be denoted by
�
(Q(0), Z(0), Q, Z, A, L
q
, S
.
47
Also observe that Si
(t) = 1, for S =�
S1(t), . . . , SI+1; t� 0�
and A(t) = t, t� 0 a.s. Obviously, it is
enough to prove (119) for any such (further) subsequence.
By appealing to the Skorohod representation theorem, we may choose an equivalent distributional
representation (which we will denote by putting a “ ˜ ” above the symbols) such that the sequence
of random processes
n⇣
˜Q
�(0), ˜Z�(0), ˜Q�
,
˜Z
�
,
˜A
�
,
˜L
�
q
,
˜S
�
⌘o
as well as the limit
⇣
˜Q(0), ˜Z(0), ˜Q,
˜Z,
˜A,
˜L
q
,
˜S
⌘
are defined on a new probability space, say (⌦, G, P ), so that P -a.s.
⇣
˜Q
�(0), ˜Z�(0), ˜Q�
,
˜Z
�
,
˜A
�
,
˜L
�
q
,
˜S
�
⌘
!
⇣
˜Q(0), ˜Z(0), ˜Q,
˜Z,
˜A,
˜L
q
,
˜S
⌘
(122)
u.o.c. as �!1. Clearly ˜S
i
(t) = 1, t� 0 a.s. for ˜S =
⇣
˜S1(t), . . . ,
˜S
I+1; t� 0⌘
, hence we drop it from
the notation for the rest of the proof. Also, ˜A(t) = t a.s. for t� 0.
By the equivalent distributional representation, the limit⇣
˜Q(0), ˜Z(0), ˜Q,
˜Z,
˜A,
˜L
q
⌘
satisfies the
fluid model equations (99)–(109) if Assumption 1 holds. Hence, it follows from (120) and Theorem 7
that there exists T > 0 such that
P
n
�
�
�
( ˜Q(t), ˜Z(t))� (q⇤, z⇤)�
�
�
> ✏
o
< ✏
0/2 (123)
for all t� T . By (122) and the equivalent distributional representation, this implies that
P
�
�
�(Q�(t), Z�(t))� (q⇤, z⇤)�
�
> ✏
< ✏
0
for all t � T and � large enough. We obtain the desired result (120) by the fact that the initial
state (in the original probability space) (Q�(0), Z�(0)) has a stationary distribution for each �. If
Assumption 1 does not hold, (123) (hence the result) follows from Proposition 3. ⇤
D.4. Proof of Theorem 3
The proof is similar to the proof of Theorem 2 hence we only present a sketch. The main di↵erence
in the proof is that we need to show that Theorem 7 is still valid under ⇡
0 when (23) holds. The
proof is similar to that of Theorem 7. It is easily checked that (150) in Step 1 and (151) in Step
2 of Theorem 7 are still valid when (23) holds. Then Step 3 is identical since ⇡
0 takes the same
actions with ⇡
�,⇤ when (153) and (154) hold. ⇤
48
E. Proof of Proposition 1
Assume that (9), (26) and (27). holds. (Throughout the proof we use the notation defined in §B.)
Recall that z
⇤(1,N) = (z⇤0 , z⇤1 , . . . , z
⇤I
) denotes the optimal solution of (10) (with sta�ng level N
and arrival rate �= 1), see (18).
As in the proof of Theorem 2, we focus on the fluid limits. First we give an explanation of how
the queueing equations are obtained under ⇡
P
. Let levels i
j
and i
j+1 be e�cient and recall that
U
ijdenote the set of ine�cient levels in between these two levels. With a slight abuse of notation,
let A
�
i
(t) denote the number of customers who are assigned to an agent in levels {i
j
}[ U
ijupon
arrival. By construction of our policy there may be at most one agent in one of the levels in U
ij.
Let
D
�
Uij(t) =
X
i2Uij
D
�
i
(t) and D
�
Uij(t) =
D
�
Uij(t)
�
.
Obviously
X
i2Uij
Z
�
i
(·)! 0 and D
�
Uij(·)! 0 (124)
u.o.c. a.s. as �!1.
Now because of the preemption procedure
�
�
�
�
�
�
Z
�
ij+1(t)�Z
�
ij+1(0)�
0
@
A
�
ij(t)�D
�
ij+1(t)�D
�
Uij(t)
i
j+1 � i
j
�
⇣
A
�
ij+1(t)�D
�
ij+2(t)�D
�
Uij+1(t)⌘
i
j+2 � i
j+1
1
A
�
�
�
�
�
�
2.
Using this condition and (124), one can show that the fluid limits of ⇡
P
satisfy the following
equations in addition to (99) and (104)–(109); for i /2F
Z
i
(t) = 0, for all t� 0. (125)
For ij
2F ,
Z
ij(t) = Z
ij(0)+
1
i
j
� i
j�1A
ij�1(t)�
1
i
j+1 � i
j
A
ij(t)
+d
ij+1
i
j+1 � i
j
Z
t
0
Z
ij+1(s)ds�
d
ij
i
j
� i
j�1
Z
t
0
Z
ij(s)ds, i
j
2F \ {0, I}, (126)
Q(t) = Q(0)+ A
I
(t)� d
I
Z
t
0
Z
I
(s)ds� �
Z
t
0
Q(s)ds+ L
q
(t), (127)X
ij2F
˙A
ij(t) = 1, (128)
˙A
ij0(t) = 0, if Z
ij(t)> 0 for some j < j
0, (129)
49
where J is the number of e�cient levels. The fluid model equations (99), (104)–(109) and (125)–
(129) are similar to the fluid model equations (99)–(109), with one di↵erence; the number of agents
transitioning from one level to another as explained above. We note again that a solution {Z, Q, A}
to equations (99), (104)–(109) and (125)–(129), referred to as a fluid model solution, is di↵erentiable
almost everywhere and we refer to a point where it is di↵erentiable as a regular point.
As in the proof of Theorem 2, it is enough to show that for any fluid model solution {Z, Q, A},
given ✏> 0, there exists T large enough such that
�
�
Z
i
(t)� z
⇤i
�
�
< ✏. (130)
for t � T and i = 0, . . . , I. We also note that for i /2 F , (130) follows from Lemma 1 and (125).
We prove the result in three steps. We mainly focus on the case when there are two basic levels,
denoted by i
j
and i
j+1, 0< i
j
< i
j+1 < I and d1N < 1< d
I
N and comment on other cases at the
end of the proof. Throughout the proof we only consider regular points of the fluid model solution.
Step1 (when i < i
j
): We first show that (130) holds for i < i
j
. The proof is similar to the
proof of Step 1 of Theorem 7 using an induction argument. For i= 0, the argument in Step 1 of
Theorem 7 can be repeated verbatim. Assume that ik
2F with 0< i
k
< i
j
and there exits t0 such
that Z
i
0(t) = 0 for all t� t
0 and i
0< i
k
. We show that there exists T > t
0 such that Z
ik(t) = 0 for
t � T , completing the induction argument. Because i
k
is the lowest nonempty level then at any
regular point t > t
0
˙Z
i
0(t) = 0 for all i0 < i
k
.
Therefore by (99) and (126), ˙A
ik�1(t) = (i
k
� i
k�1)dik Zik(t). Also by (126)–(129)
˙Z
ik(t) =
�1+ d
ikZ
ik(t)+ d
ik+1Z
ik+1(t)
i
k+1 � i
k
1
I
⇣
�1+ d
ik+1(N � Z
ik(t))⌘
, (131)
where the inequality follows from the fact that i
k+1 � i
k
I and i
k
, i
k+1 2F . Because i
k
< i
j
and
so i
k+1 i
j
, (131) implies that, when there are two basic levels,
˙Z
ik(t)�
1
I
(1� d
ijN)< 0,
whenever Zik(t)> 0 for a regular point t. Therefore, there exists T large enough such that
Z
i
(t) = 0 for all t� T and i < i
j
,
completing the proof of (130) for i < i
j
.
50
Step 2 (When i > i
j+1): Next we focus on i > i
j+1 and prove by induction that (130) holds for
i > i
j+1. As in the second step of proof of Theorem 7, Q(t) = 0 for t > T
0 :=M/(1� d
I
N). Assume
that ZI
(t)> 0 for a regular point t > T
0. If Zi
(t)> 0 for some i < i
J�1, then by (127) and (129)
˙Z
I
(t)�
d
I
Z
I
(t)
I � i
J�1. (132)
If Zi
(t) = 0 for all i < i
J�1, then˙Z
i
(t) = 0 for all i < i
J�1 for any regular point t by (126)–(129),
hence
˙Z
I
(t)1� d
iJ�1Z
iJ�1(t)� d
I
Z
I
(t)
I
1� d
iJ�1N
I
. (133)
By (132) and (133) there exists TI
> 0 such that ZI
(t)< ✏ for t� T
I
.
Now consider J > k > j+1 and assume that there exits Tik+1
such that, for t� T
ik+1,
Z
i
(t)1
4Imaxj
{d
j
}
✏ (134)
for i > i
k
and that
✏>
d
ik+1N
4Idij+1
. (135)
Assume also that Z
ik(t)> I✏/d
ikfor a regular point t. If Z
i
(t)> 0 for some i < i
k
� 1, then by
(126), (129), (134) and (135)
˙Z
ik(t)�
d
ikZ
ik(t)
i
k
� i
k�1+
d
ik+1Z
ik+1(t)
i
k+1 � i
k
<�✏/2. (136)
If Zi
(t) = 0 for all i < i
k�1, then˙Z
i
(t) = 0 for all i < i
k�1 for any regular point t, hence by (126),
(129), (134) and (135)
˙Z
ik(t)
1� d
ik�1Z
ik�1(t)� d
ikZ
ik(t)
(ik
� i
k�1)+
d
ik+1Z
ik+1(t)
i
k+1 � i
k
�✏/4. (137)
By (136) and (137) there exists T
ik> 0 such that Z
ik(t)< ✏ for t� T
ikcompleting the induction
argument.
Step 3 (ij
and i
j+1): Now we are ready to finish the proof by focusing on i
j
and i
j+1. We show
that given ✏ there exists T large enough such that
�
�
Z
ik(t)� z
⇤ik
�
�
< ✏, for k= j, j+1.
The proof is similar to the case with U
ij= ; in the proof of Theorem 7. Fix ✏> 0 and given ✏
0> 0
choose T (✏0) so that (153) and (154) hold. Existence of such T (✏0) is guaranteed by the first two
steps. Then by (126), (153) and (154), if Zij(t) = z
⇤ij+ ✏, for ✏> ✏ and t > T (✏0)
˙Z
ij(t)�
1
i
j+1 � i
j
⇣
(dij+1
� d
ij)✏+ d
ij+1
⌘
✏
0
51
Similarly, if Zij(t) = z
⇤ij� ✏
˙Z
ij(t)�
1
i
j+1 � i
j
⇣
(dij+1
� d
ij)✏� d
ij+1✏
0⌘
For ✏0 <⇣
d
ij+1� d
ij
⌘
/(4dij+1
)✏, this gives the desired.
Other cases: The other cases are handled in a way similar to that described at the end of the
proof of Theorem 7. ⇤
F. Proof of Theorem 4
Fix ✏> 0 that satisfies
✏<
�
2Idm
(PAb
I
d
I
_ 1), (138)
where dm
=max1i1,i2,iI
⇣
di�di2
di1�di2
⌘
� 1 and consider the policy ⇡
⇤(✏). By (6), (42), Theorem 8 and
(117) it is enough to show that
limsupT!1
lim�!1
�
�
Z
�(T )� z
⇤(1,N)�
�
< 2dm
✏. (139)
Note that by Proposition 1, there exist � and T large enough such that for the virtual system
�
�
Z
�(T )� z
⇤(1,N)�
�
< ✏.
Therefore, if z⇤ijk
(1,N)> ✏, for k= 1,2, then the result follows from Theorem 2.
Now assume that z⇤ijk
(1,N)< ✏ for k= 1 or k= 2. If ij1 = I then we have that all the agents are
in states I � 1 and I in the fluid model as we argue next. Because d
I
> d
I�1, using the first step
of the proof of Theorem H.1, we have for large enough t, ZI�1(t) d
m
✏ by definition of dm
and
Z
I
(t) =N � Z
I�1(t). Hence (139) follows.
Now assume that z
⇤ij2
(1,N) < ✏ and let j1 denote lowest indexed level greater than i
j1 that
is not in N . Note that as in the proof of Theorem H.1, Zj1(t) d
m
✏ (by definition of dm
) and
Z
ij1(t) � N � Z
j1(t)� d
m
✏, Zi
(t) = 0 for i i
j1 andP
i>j1Z
i
(t) < d
m
✏ for t large enough. Hence
(139) follows.
Now assume that z⇤ij1
(1,N)< ✏ and let j2 denote lowest indexed level greater than i
j2 that is not
in N . By steps 1 and 2 of Theorem H.1, Zi
(t) = 0 for i < i
j2 � 1, and Z
ij2�1(t)< d
m
✏ by definition
of dm
andP
i>j2Z
i
(t) < d
m
✏/I, for t � T and T large enough. Note that because j2 denotes the
smallest indexed level after ij2 that is not in N , all levels in {i
j2 +1, . . . , j2 � 1} (if there are any)
are in N . In this case (139) follows similarly to the third step of the proof of Theorem H.1 as we
explain next. Set T = 0 for notational simplicity. If Zij2
�1(t)+ Z
ij2(t)>N�d
m
✏ for any t the result
52
readily follows. Therefore assume that Zij2
�1(0)+ Z
ij2(0)<N �d
m
✏. Then, it is easy to show using
the fluid model equations that either Z
ij2�1(s) = 0 or Z
ij2�1(s) + Z
ij2(s) > N � d
m
✏ for s large
enough. In the latter case the desired result follows from the preceding argument so assume that
the former holds. Also assume that Z
ij2(t)<N � d
m
✏ for all t� s as otherwise the result follows
again from the preceding argument. Then it is easy to check that Zij2
�1(t) = 0 and similar to (159)
and (160) we have for t� s that
Z
ij2(t)� Z
ij2(s) (140)
and for any ✏
b
> 0 there exists finite T (✏b
) large enough such that
d
b
(t)� (1� ✏
b
)_⇣
d
b
(s)^ 1⌘
(141)
for db
(t) = d
ij2Z
ij2(t)+ d
j2Zj2(t) and t� T (✏b
). The rest of the proof follows similarly to the third
step of the proof of Theorem H.1. ⇤
G. Proofs of the results in Section 5
Proof of Theorem 5: The proof follows from Theorems 1 and 2. First it is easily checked that
N
⇤(�n
, p
Ab)
n
= �
n
N
⇤(1, pAb).
Therefore,
limn!1
N
⇤(�n
, p
Ab)
n
=N
⇤(�, pAb). (142)
Assume that there exists an asymptotically feasible sequence N
n such that
limsupn!1
N
⇤(�n
, p
Ab)
N
n
> 1. (143)
Then there exists a subsequence (for simplicity of notation, we still use index n), along which
limn!1
N
⇤(�n
, p
Ab)
N
n
> 1 (144)
and
limn!1
N
n
n
=N (145)
for some N . Also, by (143) and (144), we have
N <N
⇤(�, pAb). (146)
53
Note that PAb (�,N)>P
Ab (�,N⇤(�, pAb)), otherwise N⇤(�, pAb) =N and (146) cannot hold. By
Theorems 1 and 2, for any non-idling policy ⇡
n, if the sta�ng level is equal to N
n in the nth
system, we have
lim infn!1
P
Ab,⇡
n(�n
,N
n)� limsupn!1
P
Ab,⇡
⇤(�n
,N
n) = P
Ab (�,N)>P
Ab
�
�,N
⇤(�, pAb)�
. (147)
Hence, Nn cannot be feasible and in turn (143) cannot hold. Also,
limn!1
P
Ab,⇡
⇤(�n
,N
⇤(�n
, p
Ab)) = P
Ab(�,N⇤(�, pAb)) p
Ab
, (148)
where the equality follows from Theorem 2, (38) and (142) and the inequality follows from the fact
that N⇤(�, pAb) is an optimal solution (hence a feasible solution) of the sta�ng LP. ⇤Proof of Theorem 6: Fix � > 0. If p
Ab = P
Ab
1 the result follows from Lemma 1 and (42) and
Theorem 4 by setting ✏= �/2. Note that if pAb
>P
Ab
1 then �� (1+ �)N⇤(�, pAb)d1 for some �� �> 0
by Lemma 2 since both levels 1 and I are assumed to be e�cient. For notational simplicity we set
N
⇤ =N
⇤(�, pAb). Define
✓= inf��(1+�)N⇤
d1
n
P
Ab (�,N⇤)�P
Ab
⇣
�, (1+ �)N⇤⌘o
. (149)
By Lemma 1 and (42), ✓> 0. Choose ✏> 0 as in Theorem 4 so that (29) holds with ✓. The result
then follows from Theorem 4. ⇤
H. Proofs of supplemental results
In this section we prove Theorem 7 in §H.1, Proposition 3 in §H.2 and Theorem 8 in §H.3. (Through-
out this section we use the notation defined in §B.)
H.1. Proof of Theorem 7
Fix N and let i⇤(1,N) = {i
j
, i
j+1} denote the set of basic levels. We mainly focus on the case when
0< i
j
< i
j+1 < I and d1N < 1< d
I
N . We describe how the proof can be extended to other cases at
the end. In addition we focus mainly on the case when N = ; and extend the proof using Lemma 6
below when it is not. Let (Q, Z, A, L
q
) denote a fluid model solution. Throughout the proof we use
the following fact that if Zi
(t) = 0 at a regular point t then ˙Z
i
(t) = 0, since Zi
attains its minimum
at t.
Fix ✏> 0. We prove (114) in three steps.
1. We first show that there exists T1 � 0 such that Zi
(t) = 0 for t� T1 and i i
j
� 1.
2. Then we show that given ✏> 0 there exists T2 � T1 such that Zi
(t)< ✏ for t� T2 and i > i
j+1.
54
3. In the last step, we show that given ✏> 0 there exists T3 � T2 such that�
�
Z
i
(t)� z
⇤i
�
�
< ✏ for t� T3
and for i= i
j
, i
j
+1 . . . , ij+1, completing the proof.
Because (Q, Z, A, L
q
) is di↵erentiable almost everywhere, when we take the derivatives with
respect to t we only consider the regular points of (Q, Z, A, L
q
) throughout the proof without loss
of generality.
Step 1. Let
f(t) =
ij�1X
i=0
(ij
� i)Zi
(t).
Assume that f(t)> 0. Note that by (99)–(109), for any regular point t we have
f(t) =�1+
ijX
i=1
d
i
Z
i
(t)<�1+ d
ijN <��, (150)
where � = (dij+1
� d
ij)z⇤
ij+1and the last inequality follows from (18), and the fact that there are
two basic levels. Hence Z
i
(t) = 0, i i
j
� 1, for t� T1 :=N/�.
Step 2. Now we consider i > i
j+1. By (102), (106) and (107), if Q(t)> 0, we have
˙Q(t) = 1� d
I
N � �Q(t).
Hence, Q(t) = 0 for t > T
0 :=M/(1� d
I
N). Let
f(t) =I
X
i=ij+1+1
(i� i
j+1)Zi
(t).
By our assumption N = ; and (99)–(109), if f(t)> ✏ for t > T
0, then
f(t)��, (151)
where � =minn
d
ij+1+1✏/I,�1+ d
ij+1+1N
o
, which is positive by (14). By our assumption N = ;,
we conclude that
Z
i
(t)< ✏ (152)
for t� T2 := T
0 +N/� and i > i
j+1.
Step 3. To conclude the proof we need to show that (114) is satisfied (for a reselected ✏) by Z
ij,
Z
i
for i2 U
ij(if U
ij6= ;) and Z
ij+1. By the first two steps, we can assume that for any ✏
0> 0 there
exists a finite T (✏0)> 0 such that for t� T (✏0), Q(t) = 0,
Z
i
(t) = 0 for i i
j
� 1 (153)
55
and
Z
i
(t) ✏
0 for i� i
j+1 +1. (154)
Assume that U
ij= ;. Fix ✏, ✏
0> 0 so that
✏
0<
⇣
d
ij+1� d
ij
⌘
/(4dij+1
)✏. (155)
Recall that by Lemma 3
d
ij+1> d
i
(156)
for all i < i
j+1 otherwise i
j+1 cannot be an e�cient level. By (100), (105), (107) and (153)
˙Z
ij(t) =�(1� d
ijZ
ij(t))+ d
ij+1Z
ij+1(t)
Fix ✏> 0. Then by (105), (153) and (154), if Zij(t) = z
⇤ij+ ✏, for ✏> ✏,
˙Z
ij(t)�(d
ij+1� d
ij)✏.
Similarly, if Zij(t) = z
⇤ij� ✏
˙Z
ij(t)� (d
ij+1� d
ij)✏� d
ij+1✏
0.
This gives the desired result (114) for i
j
, i
j+1 with T > T (✏0) + 4N/
⇣
(dij+1
� d
ij)✏⌘
by (155) and
(156).
Now assume that U
ij6= ;. Define
d
b
(t) = d
ijZ
ij(t)+ d
ij+1Z
ij+1(t).
By (153), (154), (156) and (104), given � > 0, there exists ✏
0, ✏> 0 such that if d
b
(t)� 1� 2✏, and
Z
ij(t)� z
⇤ij� 2✏ for t� T (✏0), then
Z
ij+1(t)2 [z⇤
ij+1� �, z
⇤ij+1
+ �].
Therefore, it is enough to prove that for ✏> 0 small, there exists T such that for t� T � T (✏0),
d
b
(t)� 1� 2✏ and Z
ij(t)� z
⇤ij� 2✏. (157)
For the rest of the proof fix ✏> 0 and ✏
0> 0 small so that
(d1 +1)
1�d
ij
d
ij+1
!
✏� 4�
|U
ij|+1
�
d
ij+1✏0. (158)
56
From here on we only consider t� T (✏0) so take T (✏0) = 0 for notational simplicity and assume that
(153) and (154) hold for t� 0.
We begin the proof in this case with the following preliminary results. For all t� 0 and ✏> 0
Z
ij(t)� Z
ij(0)^
⇣
z
⇤ij� ✏
⌘
(159)
and for any ✏
b
> 0 there exists finite T (✏b
) large enough such that
d
b
(t)� (1� ✏
b
)_⇣
d
b
(0)^ 1⌘
(160)
for t� T (✏b
).
Proof of (159): By (158) we have
✏�
2dij+1
d
ij+1� d
ij
✏
0 (161)
Note that because ij
�1 has priority over i� i
j
and i
j
has lower priority than those levels in U
ij, if
Z
ij(t) z
⇤ij� ✏, either there exists i2 U
ijwith Z
i
(t)> 0 or by (158), db
(t)� 1 and Z
i
(t) = 0 for all
i 2 U
ij. It is easy to show that if d
b
(t)� 1 and Z
i
(t) = 0 for all i 2 U
ij, then t cannot be a regular
point. Therefore, by (100) and (105)
˙Z
ij(t)� 0 (162)
whenever Zij(t) z
⇤ij� ✏. This gives (159).
Proof of (160): Now assume that
d
b
(t) 1. (163)
Let i be the highest non-empty level in U
ij. Then, for all i < i
0< i
j+1,
˙Z
i
0(t) = 0,
by (100)–(105) since Z
i
0 attains a minimum at t. Therefore, if (163) holds, by (100)–(105)
˙Z
ij+1(t)�
1� d
b
(t)
|U
ij|
and ˙Z
ij(t)� 0,
where |U
ij| is the cardinality of the set U
ij.
Note also that if for all the levels i 2 U
ij, Z
i
(t) = 0 (implying ˙Z
i
(t) = 0) then when (163) holds,
by (100)–(105),
˙Z
ij+1(t)�
1� d
b
(t)
|U
ij|
and ˙Z
ij(t) =�
1� d
b
(t)
|U
ij|
.
Therefore, db
(t) is increasing if db
(t) 1 and so (160) holds.
We consider the following three cases separately to complete the proof.
57
(A) Z
ij(0)� z
⇤ij+ ✏
(B) Z
ij(0) z
⇤ij+ ✏, d
b
(0)� 1
(C) Z
ij(0) z
⇤ij+ ✏, d
b
(0) 1
Case (A): Assume that Zij(0)� z
⇤ij+ ✏. Then, by (160) for t large enough d
b
(t)� 1� ✏ for t� T .
This with (159) gives (157).
Case (B): Assume that Z
ij(0) z
⇤ij+ ✏ and d
b
(0)� 1. By (160), db
(t)� 1, for all t� 0. Thus,
to prove (157), it is enough to show that there exists T large enough such that Z
ij(t) � z
⇤ij� 2✏
for t� T . But by (162), this can be proved by showing that there exists a t > 0 such that Zij(t)�
z
⇤ij� 2✏.
Assume that no such t exists, that is,
Z
ij(t) z
⇤ij� 2✏ for all t� 0. (164)
Let
✏1(t) = z
⇤ij� Z
ij(t) (165)
and
✏2(t) =1� d
ijZ
ij(t)� d
ij+1z
⇤ij+1
d
ij+1
=d
ij
d
ij+1
✏1(t). (166)
Because d
b
(t)� 1 for t� 0, Zij+1
(t)� z
⇤ij+1
+ ✏2(t).
Let �(t) = Z
ij+1(t)� z
⇤ij+1
� ✏2(t). By (161), (165), and (166),
�(t)+X
i2Uij
Z
i
(t)�N � ✏
0� Z
ij(t)� z
⇤ij+1
� ✏2(t) = ✏1(t)� ✏2(t)� ✏
0> 0.5
1�d
ij
d
ij+1
!
✏. (167)
Now define the Lyapunov function
f(t) = (ij+1 � i
j
) Zij+1
(t)+X
i2Uij
(i� i
j
) Zi
(t).
Note that f(0)< IN and that f(t) = 0 implies
Z
ij+1(t)+
X
i2Uij
Z
i
(t) = 0. (168)
If f(t)> 0, we have (assuming (164) and d
b
(t)� 1 for all t� 0)
f(t) = (ij+1 � i
j
) ˙Z
ij+1(t)+
X
i2Uij
(i� i
j
) ˙Z
i
(t)
58
1� d
ijZ
ij(t)� d
ij+1Z
ij+1(t)�
X
i2Uij
d
i
Z
i
(t)+ (|Uij|+1)d
ij+1✏0 (169)
�d
ij+1�(t)�
X
i2Uij
d
i
Z
i
(t)+ (|Uij|+1)d
ij+1✏0 (170)
�0.5d1
1�d
ij
d
ij+1
!
✏+(|Uij|+1)d
ij+1✏0 (171)
�0.25d1
1�d
ij
d
ij+1
!
✏
:=��, (172)
where (169) follows from (100)–(105) and the fact that ij
� 1 has priority over all the other levels
i� i
j
� 1, (170) follows from the definition of �(t), (171) follows from (167) and (172) follows from
(158). Therefore, for T large enough f(t) = 0 for t� T and so (168) holds for all t� T . However,
this contradicts the fact that db
(t)� 1 for all t� 0. Hence, Zij(t)� z
⇤ij�2✏ for some t2 [0, (IN)/�],
giving (157) in this case.
Case (C): Assume that, Zij(0) z
⇤ij+ ✏ and d
b
(0) 1. By (160), we can assume without loss of
generality that d
b
(t)� 1� ✏ for t large enough. If db
(t)� 1, the proof is complete by Case (B) so
assume that 1� ✏ d
b
(t)< 1.
Also, if Zij(t) > z
⇤ij� 2✏ for some t, (157) follows from (159). Therefore, assume that Z
ij(t) <
z
⇤ij� 2✏ for all t � 0. Then, because d
b
(t) < 1, Zij+1
(t) < z
⇤ij+1
+ ✏2(t) for ✏2 defined as in (166).
Hence
X
i2Uij
Z
i
(t)� ✏1(t)� ✏2(t)� ✏
0> 0.5
1�d
ij
d
ij+1
!
. (173)
Because d
b
(t)< 1, by (100)–(105), (173), and the fact that ij+1 � 1 have higher priority than i
j
and i
j+1,
X
i2Uij
(i� i
j
) ˙Z
i
(t)�d1
X
i2Uij
Z
i
(t)�0.5d1
1�d
ij
d
ij+1
!
✏
:=��0.
Therefore, for T �N/�0
X
i2Uij
Z
i
(T ) = 0.
Hence,
Z
ij+1(T ) =N � Z
ij(T )� ✏
0.
However, because Z
ij(t) < z
⇤ij� 2✏ for all t � 0, this implies d
b
(T ) > 1 for ✏
0 small, contradicting
our earlier assumption and completing the proof by Case (B).
59
Other cases: If there is only one basic level 1< i
j
< I then we have
Z
i
(t) = 0, for i < i
j
� 1
and t � T1 := N/�, where � = (dij+1
� d
ij)z⇤
ij+1, using an argument similar to that in Step 1.
Therefore, for t� T1, by (100) and (105)
˙Z
ij�1(t)�
⇣
d
ij� d
ij�1
⌘
˙Z
ij�1(t).
Hence
Z
ij�1(t)< ✏ (174)
for t� T2 :=N/
⇣⇣
d
ij� d
ij�1
⌘
✏
⌘
+T1. If 1< d
I
N , the result then follows from Step 2.
If 1 = d
I
N , then there is only one basic level I. By (100) and (102), t� T
0 :=M/(�✏), we have
Q(t)< ✏ and by (174), ZI
(t)>N � ✏, proving the result for t� T
0 large enough.
If 1> d
I
N , we have by (150) that ZI
(t) =N , for all t > T
0 :=N/(1� d
I
N). By (113) and (99)–
(109), this implies that�
�
Q(t)� q
⇤�
�
< ✏ for all t > T1 := T
0 +M/(�✏) + q
⇤/(�✏), proving (114) if
1> d
I
N .
Finally we consider the cases when there are two basic levels i
j
and i
j+1 but either i
j
= 0 or
i
j+1 = I. If ij
= 0, then i
j+1 = 1, since 1 is an e�cient level by definition. Then the result is proved
using (152) as in the case with U
ij= ; in Step 3. If i
j+1 = I then the result is proved in the same
way as above except that we do not need Step 2 and do not require (154) in Step 3 anymore. ⇤If N 6= ;, the second step in the proof needs to be modified. The result in that case follows from
the following lemma.
Lemma 6. Assume that the conditions of Theorem 7 hold and that ij+1 < I. Then for any ✏> 0
there exists t2 > 0 such that
Z
i
(t)< ✏ for t� t2 and i > i
j+1.
Proof of Lemma 6: Assume that the conditions of Theorem 7 hold and that i
j+1 < I. Let {⌫
i
}
denote a sequence of nonnegative finite constants {⌫i
} for i� i
j+1 +1 such that
• ⌫
i
= 1 if i /2N ,
• If i2N
⌫
i
�
✓
d
⇤
d
i
_ 1
◆
⌫
i
0, for all i0 > i, (175)
where d
⇤ =min{di
0 : i0 � i
j+1 +1, i0 /2N}.
60
Also, let {⇢i
} be defined for i� i
j+1 +1 recursively as follows;
⇢
i
= ⌫
i
+ ⇢
i�1,
where ⇢
ij+1= 0. Define
f(t) =I
X
i=ij+1+1
⇢
i
Z
i
(t).
Observe that, given ✏> 0, it is enough to show that there exists T > 0 such that
f(t)< ✏, for all t� T .
Fix ✏ and assume that f(t)> ✏. This impliesP
I
i=ij+1+1 Zi
(t)> ✏/⇢
I
. Recall that we assume I /2N .
Case 1: Assume that Zn
(t)> 0 for some n< i
j+1, then by (100) and (105)
f(t) =I
X
i=ij+1+1
⇢
i
˙Z
i
(t)I
X
i=ij+1+1
⇢
i
�
�d
i
Z
i
(t)+ d
i+1Zi+1(t)�
�d
⇤I
X
i=ij+1+1
Z
i
(t)<�d
⇤✏/⇢
I
.
Case 2: Now assume that Zn
(t) = 0 for all n< i
j+1, then for a regular point t,
˙Z
ij+1�1(t) = 0, (176)
because Zij+1�1(t) = 0 (note that this is only possible if 1� d
ij+1Z
ij+1(t), which we assume is true),
andP
I
i=ij+1Z
i
(t) =N . Note that (176) implies
˙A
ij+1�1(t) = d
ij+1Z
ij+1(t). (177)
Because 1 < d
I
N , we can assume without loss of generality that Z
I
(t) <N , for all t � 0, and so
there exists i < I with Z
i
(t)> 0, by (104).
Case 2(a): Assume that there exists a level n such that ij+1 n < I, Z
n
(t)> 0 and n /2N
0 and
let i /2N
0 be the lowest such level at time t. Note that i+ 1 /2N . By the definition of the policy
and (105), if i0 2N
0 and i
0 i
˙A
i
0(t) = 0. (178)
Also, if i0 /2N
0 and i
0 i, we claim that
˙A
i
0(t) = d
i
0+1Zi
0+1(t). (179)
61
To prove this first note that ˙Z
i
0(t) = 0, for a regular point t. Therefore, by (100), it is enough
to show that ˙A
i
0�1(t) = 0. If Z
i
0�1(t) > 0, then i
0� 1 2 N
0 and by (178), ˙A
i
0�1(t) = 0. Assume
that Z
i
0�1(t) = 0 and so ˙Z
i
0�1(t) = 0. If Zi
0�2(t) > 0, therefore i
0� 2 2 N
0, then ˙A
i
0�2(t) = 0 by
(100). Because ˙Z
i
0�1(t) = 0, this implies ˙A
i
0�1(t) = 0 by (100). If Zi
0�2(t) = 0, we repeat the same
argument until we reach to level ij+1 or the highest level i0 � k before i
0 with Z
i
0�k
(t)> 0. Also,
˙A
i
0(t) = 0, for i0 � i+1 (180)
by (105). Therefore by (100) and (103)
˙Z
i+k
(t) =�d
i+k
Z
i+k
(t)+ d
i+k+1Zi+k+1(t), for k= 2, . . . , I � i� 1, (181)
˙Z
I
(t) =�d
I
Z
I
(t). (182)
For ij+1 k i, by (100)–(104),
˙Z
k
(t) = ˙A
k�1(t)+ d
k+1Zk+1(t)�˙A
k
(t)� d
k
Z
k
(t) (183)
and
˙Z
i+1(t) =˙A
i
(t)+ d
i+2Zi+2(t)� d
i+1Zi+1(t). (184)
By (178)–(180) and (103)
˙A
i
(t) = 1�X
k2[ij+1+1,i]k/2N
d
k
Z
k
(t)� d
ij+1Z
ij+1(t). (185)
Then, by (181)–(184)
f(t) =I�1X
k=ij+1+1
⇢
k
�
d
k+1Zk+1(t)� d
k
Z
k
(t)�
� ⇢
I
d
I
Z
I
(t)+i
X
k=ij+1+1
⇢
k
⇣
˙A
k�1(t)�˙A
k
(t)⌘
+ ⇢
i+1˙A
i
(t)
(a)
�
I
X
k=ij+1+1
⌫
k
d
k
Z
k
(t)+i
X
k=ij+1+1
⌫
k
˙A
k�1(t)+ ⌫
i+1
0
B
B
@
1�X
k2[ij+1+1,i]k/2N
d
k
Z
k
(t)� d
ij+1Z
ij+1(t)
1
C
C
A
(b)= �
I
X
k=ij+1+1
⌫
k
d
k
Z
k
(t)+X
k2[ij+1+1,i]k/2N
⌫
k
d
k
Z
k
(t)+ 1�X
k2[ij+1+1,i]k/2N
d
k
Z
k
(t)� d
ij+1Z
ij+1(t) (186)
(c)= 1� d
ij+1Z
ij+1(t)�
I
X
k=ij+1+1
⌫
k
d
k
Z
k
(t)
(d)
1� d
ij+1Z
ij+1(t)� d
⇤(N � Z
ij+1(t))
(e)
�c✏,
62
for some constant c > 0, where (a) follows from (180), (185) and algebraic manipulations, (b)
follows from the fact that i+1 /2N , so ⌫
i+1 = 1, (178) and (179), (c) follows from simple algebraic
manipulations, (d) follows from the definition of ⌫k
’s and the fact thatP
I
i=ij+1Z
i
(t) =N , (e) follows
from the definition of d⇤.
Case 2(b): Now assume that Z
n
(t) = 0 for all n < i
j+1 and Z
k
(t) = 0 for all k � i
j+1 and k /2
N
0\ {I}. Let i 2N
0 be the highest level with Z
i
(t)> 0. We note that (178) and (179) still hold.
Also, (183) holds by (100).
By definition of the policy Z
k
(t) = 0 for all i+1 k I � 1 and so
˙Z
k
(t) = 0, for i+1 k I � 1, (187)
because t is a regular point. Therefore, by (100)–(104)
˙Z
i
(t) = ˙A
i�1(t)�˙A
i
(t)� d
i
Z
i
(t), (188)
and
˙Z
I
(t) = ˙A
i
(t)� d
I
Z
I
(t), (189)
where by (177)-(179) and (187)
˙A
i
(t) =
1� d
ij+1Z
ij+1(t)� d
I
Z
I
(t)�P
k2[ij+1+1,i]k/2N
d
k
Z
k
(t)
I � i
. (190)
We note that ˙A
i
(t)� 0 otherwise t cannot be a regular point.
Then, similar to case Case 2(a),
f(t)(a)
i
X
k=ij+1+1
⇢
k
�
d
k+1Zk+1(t)� d
k
Z
k
(t)�
� ⇢
I
d
I
Z
I
(t)+i�1X
k=ij+1+1
⇢
k
⇣
˙A
k�1(t)�˙A
k
(t)⌘
+ ⇢
i
˙A
i�1(t)+ (⇢I
� ⇢
i
) ˙A
i
(t)
(b)
�
X
k2[ij+1+1,i]k2N
⌫
k
d
k
Z
k
(t)� ⇢
I
d
I
Z
I
(t)+
P
I
k=i+1 ⌫k
I � i
0
B
B
@
1� d
ij+1Z
ij+1(t)�
X
k2[ij+1+1,i]k/2N
d
k
Z
k
(t)
1
C
C
A
(c)
�
X
k2[ij+1+1,i]k2N
⌫
k
d
k
Z
k
(t)� ⇢
I
d
I
Z
I
(t)+ ⌫
i+1
0
B
B
@
1� d
ij+1Z
ij+1(t)�
X
k2[ij+1+1,i]k/2N
d
k
Z
k
(t)
1
C
C
A
(d)
⌫
i+1
�
1� d
ij+1Z
ij+1(t)� d
⇤(N � Z
ij+1(t))�
(e)
�c✏,
for some constant c > 0, where (a) follows from (179), (183), (187), (188), and (189), (b) follows
from (190) and algebraic manipulations similar to that in (186), (c) and (d) follow from (175), (e)
follows from the fact thatP
I
k=ij+1Z
k
(t) =N and from the definition of d⇤. ⇤
63
H.2. Proof of Proposition 3
Assume that the conditions of the proposition hold. We prove the result by contradiction. Assume
that (116) does not hold. Then we can find a subsequence, denoted again by �, such that
limn!1
P
�
�
�(Q�(T ), Z�(T ))� (q⇤, z⇤)�
�
> ✏
> 2✏.
We next prove that no such subsequence exists. Let ij
denote the single basic level in i
⇤(1,N). For
simplicity we only consider the case when i
j
< i
J�1, but the proof is similar for other cases. We
note that�
Z, Q, A, L
q
�
satisfies the fluid model equations (99)–(104) and (106)–(109). It may not
satisfy (105) because i
⇤(1,N) has only one element. First assume that
lim infn!1
�
�
i
⇤(�,N�)�
�= 1. (191)
Then the result follows from Theorem 7, because along a subsequence of {n} Assumption 1(iii)
holds.
If (191) does not hold, then at least one the following does:
limsup�!1
1⇣
�
⇤ij�1
(�,N�)> 0⌘
= 1 or limsup�!1
1⇣
�
⇤ij+1
(�,N�)> 0⌘
= 1 (192)
by Lemma 1. We focus on the case when the former holds; the proof for the case when only the
latter holds is similar.
If the first equation in (192) holds, then along a subsequence �
0 the limit is attained, hence
the proposed policy is ⇡⇤(ij�1, ij) (although i
⇤(1,N) has a single level) for �0 large enough. Fix a
sample path ! 2⌦ such that (115) holds and Q(0)<M .
From the first two steps of the proof of Theorem 7, there exists T > 0 such that
Z
i
(t) = 0, for i= 0,1, . . . , ij�1 � 1 (193)
and
Z
i
(t)< ✏/I, for i= i
j
+1, . . . , I. (194)
We prove below that there exists T1 >T such that
Z
i
(t)< ✏/I, for i= i
j�1, . . . , ij � 1 (195)
for t > T1. By (104) and (193)–(195), for t > T1
Z
ij(t)>N � ✏. (196)
64
Note that (196) holds for a set of sample paths with probability greater than 1� ✏, giving the
desired result.
We prove (195) to complete the proof. Let
f(t) =
ij�1X
i=ij�1
(ij
� i)Zi
(t).
If f(t)> �, then by (105) (recall that ⇡⇤(ij�1, ij) is the policy for the case with two basic levels i
j�1
and i
j
along this sequence), (100), (103), (104), (105), (193) and (194)
f(t) =�1+
ij�1X
i=ij�1
d
i
Z
i
(t)<�
⇣
d
ij� d
ij�1
⌘
�/I
2
giving (196). ⇤
H.3. Proof of Theorem 8
Consider a sequence of chat service systems that satisfies (9) under a sequence of non-idling policies
{⇡
�
}. For x2RI+2+ , define �(x) =
P
I+1i=0 xi
. Next we show that, for � large enough, for some t0,K > 0
and 0< �< 1,
supx2RI+2
+ :�(x)>K
E
x
⇥
��
Z
�(t0), Q�(t0)�⇤
�(x)< �. (197)
In other words, � is a geometric Lyapunov function with a geometric drift size 0< �< 1, drift time
t0 and exception parameter K (see Gamarnik and Zeevi (2006)). By Theorem 5 in Gamarnik and
Zeevi (2006), (197) implies that
E
�(⇡�)
⇥
��
Z
�(0), Q�(0)�⇤
<
�
�(t0)K
1� �
, (198)
where
�
�(t0) = supx
E
x
⇥
��
Z
�(t0), Q�(t0)�⇤
�(x).
BecauseP
I
i=0 Z�
i
(t)+ Q
�
i
(t)P
I
i=0 Z�
i
(t)+ Q
�(0)+ A
�(t),
�
�(t0)E
⇥
exp(A�(t0))⇤
.
By (52) in Gamarnik and Zeevi (2006) and the fact that arrivals follow a Poisson process,
limsup�!1
E
⇥
exp(A�(t0))⇤
<1
for any finite t0 > 0. Hence, (197) and (198) give tightness. To prove uniform integrability, note
that by Markov’s inequality and (198) we have
P
�(⇡�)
�
Q
�(0)>u
�
B0 exp(�u),
65
for some constant B0.
We prove (197) to complete the proof by choosing appropriate K, t0 and �. Let N� = �
�1N
�
=max
✓
1/�, sup�
N
�
◆
. (199)
Choose K = 7 and t0 = 1/�. We now compute an upper bound of Ex
⇥
�
Z
�(t0), Q�(t0)�⇤
. Let
Z
�(·) denote the total number of customers in an M/M/1 queue with arrival rate � and ser-
vice rate �. Then by a coupling argument one can show that E
x
⇥
�
Z
�(t0), Q�(t0)�⇤
N
� +
E
�
Z
�(t)|Z(0) = x
I+1
�
. It follows from the Kolmogorov equation (see Example 2 in Chapter 4.6 of
Karlin and Taylor (1975)) that for t > 0
E(Z�(t)|Z(0) = x
I+1) =1
�
(1� e
��t)+x
I+1e��t
.
When �(x) > K, we must have x
I+1 � 6 due to the choice of K and the fact thatP
I
i=0 xi
<
supn
N
�. So by (199) and the choice t0,
E(Z�(t0)|Z(0) = x
I+1) +x
I+1e��t
+x
I+1/22
3x
I+1.
Again, by (199)
supx2RI+2
+ ,�(x)>K
E
x
⇥
�
Z
�(t0), Q�(t0)�⇤
(x)
N
� + 23x
I+1
N
� +x
I+1
=5
7.
Thus, (197) holds for �= 5/7. ⇤