Queuing Theory

Queuing Theory

• Queuing theory is the study of waiting in lines or queues.

Server

Server

Server

Pool of potentialcustomers

Rear of queue

Front ofqueue

Line (or queue) of customers

List of servers able to service the customers

Queuing Theory -- cont.

What do we want to know about a queuing system?

• The average or expected wait time

• The percentage of customers who experience long wait times

• The probability that a customer must wait(The probability that all servers are busy)

• The average number of customers in the queue.

• The probability that servers are idle

Simulations

• Simulation is a method of evaluation without using methmatical models such as queuing theory.

• In general, a simulation is a computer-programmed model of something.

• The best test of an OS: the real marketplace.

• The next best test: create situation similar to the real world.

• Simulations play a key role in the development of complex system such as computer networks, databases, and OS

Queueing Theory

• Customer arrive at a queueing system randomly at time (arrival times)

• Poisson arrival process: the interarrival times are distributed exponentially:

: constant average arrival rate = customer/unit time

The # of arrivals / unit time is poisson distributed with mean

t t t tn0 1 2 k k kt t 1

= interarrival times= the time between successive arrivals

P t e t( ) 1

P t e t( ) 1

y

t (time)

y = 1

P(t) =

Probability ofan arbitraryinterarrival timebeing less than t

P t e t( ) 1for large

P t e t( ) 1for small

t = 0

Example:arrival rate: one every two minutesThe rate of one customer every two

minutes = 0.5 customers per minute

P t e et t( ) . 1 1 5t P(t)-------------------------------------0 01 .3932 .6323 .7774 .8655 .918.. ..10 .993.. ..20 .99995

The probability function does not tellus when customers will arrive.It does, however, provide information about the random arrival process.

Server 1

Server C

# of customersin service

Service time

# of customersin the queue

Time spent in the queue

Total time a customer spendsin the queueing system

Total # of customers in the queueing system

Averagearrival rate

interarrivaltime

N

Ns

Nq

q

S

W

• The probability that exactly n customers will arrive in any time interval of length t is

• Let Sk denotes the service time that the Kth arriving customer requires from the system. An arbitrary service time is referred to as S, and the distribution of service time is

For random service with average service rate

e t

nn

t n

( )

!( 0, 1, 2 )

Q t P s t( ) ( )

Q t P s t e t( ) ( ) 1

Queue Disciplines• A queue discipline is the rule used for choosing t

he next customer from the queue to be serviced.

• Kendall notationA/B/c

A is the interarrival time distributionB is the service time distributionc is the number of servers

A and B may be– GI for general independent interarrival time

– G for general service time

– M for exponential interarrival or service time distribution

– D for deterministic interarrival or service time distribution

• M/M/1

• M/M/c

Traffic IntensityTraffic Intensity

• A measure of that system's ability to service its customer effectively

It is useful for determining the minimum number of identical servers a system will need in order to service its customer without its queue becoming indefinitly large or having to turn customer away.

• Ex: E(s) = 17 sec E() = 5 sec

u = 17/5 = 3.4 need at least 4 servers

uE s

EE s

( )

( )( )

arrival rate

service rate

Case StudiesA shared Laser printer

• An average of 64 requests occurring at random times during eight-hour day.

• Each request require an average of about 5 minutes to print.

• Receiving compaints from employees that they must wait nearly half an hour for their printout.

8 requests/hour = 2/15 requests/minute 12 requests/hour = 1/ 5 request/minute

Server UtilizationServer Utilization

• Server utilization is defined as the traffic intensity per server

• is the probability that a particular server is busy

• this is approximately the fraction of time that each server is in use

• For single-server system, u =

u

c c

Probability of All Server IdleProbability of All Server Idle

pi c

i

i

c

ci

c

01

0

1

1

[ !* !* *( )]

p0 1

When c= 1

(1 - 2/3 = 1/3)

Probability of All Server BusyProbability of All Server Busy

• Erlang’s C Formula:

C cp

c

c

c( , / )

! ( )

0

1

When c = 1

C c( , / ) (2/3)

Expected Number of CustomersExpected Number of Customers

• The expected number of customers in the queue:

L C cq ( , / )/( )1

When c = 1

Lq

2

1

Expected Wait TimeExpected Wait Time

• Little’s Formula

• Expected wait time

L Wq q

W C c cq ( , / )/[ ( )] 1

When c = 1

Wq

/

1

(2/15)* 10 = 4/3

[(2/3)/0.2]/(1-2/3) = 10 minutes

90th Percentile Wait Time

• 90% of the customers wait less than q ( )90

q C c c( ) ln[ ( , / )]/[ ( )]90 10 1

When c = 1

q ( )

ln( )90

10

ln(10*2/3)/(1/5 - 2/15) = 28.4 minutes

Case Study 2Master Scheduler

• Computer running simulation programs requires a lot of CPU time and scheduled on a FIFO basis.

• Generally submit about 100 programs per day

• The programs require an average of about one hour of CPU time.

100 requests/day = 100/24 = 4.2 requests/hour 1 requests/hour = 1 request/hour

Example 1• The election of the President of Stu

dent Association（學生會） is just finished in Tamkang University. There are 27,000 students and the average rate of voting is 15%. The official voting period was 5 days, from 8:00 to 17:00. The association prepared only one counter for everyone to vote and each voting requires half minute to complete. Use the equation provided to analyze the condition of student getting in-line and wait for the voting. You must explain the meaning of each equation. (ln(2.5)=0.9, ln(7.5)=2, ln(15)=2.7)

)*10ln()90(

1

1

)/,(

1

2

0

q

q

q

W

L

cC

P

Example 2• There are 36 programmers

per day, in average, come in to use the mainframe computer in the computing center. Each programmer uses the computer for about 15 minutes. Use the equation provided to analyze the usage of the computer. You must explain the meaning of each equation.

)*10ln()90(

1

1

)/,(

1

2

0

q

q

q

W

L

cC

P