A Rant on Queues

Van Jacobson

July 26, 2006

MIT Lincoln LabsLexington, MA

vj-ll-jul06

• Unlike the phone system, the Internet supports communication over paths with diverse, time varying, bandwidth.

• This means we often have to connect a fire hose to a soda straw.

2

vj-ll-jul06

• Unlike the phone system, the Internet supports communication over paths with diverse, time varying, bandwidth.

• This means we often have to connect a fire hose to a soda straw.

• This kind of plumbing needs an adapter. The adapter is called a queue.

2

vj-ll-jul06

How a real queue works

3

vj-ll-jul06

Sender injects a window’s worth of packets

Sender Receiver

4

vj-ll-jul06

Packets reach high to low bandwidth transition

Sender Receiver

5

vj-ll-jul06

First ack returns and releases next data packet

Sender Receiver

6

vj-ll-jul06

Steady-state reached

Sender Receiver

7

vj-ll-jul06

• The amount of data that has to be in transit to run at 100% utilization is the bottleneck bandwidth times the sender-receiver-sender round-trip delay (this is called the bandwidth*delay product).

• The bottleneck has to have this much buffer to handle the start up transient.

• What happens if it doesn’t?

8

vj-ll-jul06

•••••••••••••••••••••••••••••••••••••••

••• ••••• ••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••

••••••••••••••••••••••••••••••••••••••••••••••••••••• •••••••••••

••••••••••••••••••• ••••

••••••••••••••••••••••••••••

••••••••••••••••••••••••••••••••••

••••••••••••• ••••••••••••••••••••••••••••••••••••••••••••••••••••••••••

••••••••••••••••••••••••••••••••••••••••••

•••••••••••••••••••••••••••••• •••••••

••••••••••••••••••••• •••••••

••••••••••••••••••••••••••••••••••

•••••••••••••••••••••••••••••••••••

•• ••••••••••••••••••••••••••••••••••••••••••••••••••• ••••••••••

•••••••••••••••••••••••••••••••• •••••

•••••••••••••••••••••••••• •••••

•••••••••••••••••••••••••••••••••

••••••••••••••••••••••••••••••••••••••••

•••••••••••••••••••••••••••••••••••••• ••••••••••••••

••••••••••••••••••••••••••••••••••••••

••••••••••••••••••••••••••••••••••••••••••••

Satnet Test -- Dec 11, 1988

Time (sec)

Seq

uenc

e N

umbe

r (K

B)

0 200 400 600 800 1000 1200 1400

020

4060

8010

012

014

016

0

(averaged 100 Bytes/sec on a 8 KByte/sec link)

9

vj-ll-jul06

•• ••••••••••••

••

•

••

••••••••••••

••••••••••• •••

•••••••••••

•

•••••

•• •••• •••

•••••••••

•••••••••• •••

••••• •••

••••••••••

•

Satnet Test -- Dec 11, 1988 (Detail)

Time (sec)

Seq

uenc

e N

umbe

r (K

B)

100 120 140 160 180 200

1015

2025

3035

40

10

vj-ll-jul06

How does the queue behave vs. time?

Sender Receiver

11

vj-ll-jul06

Queue behavior at the fast-to-slow transition

Time

Que

ue le

ngth

12

vj-ll-jul06

Queue behavior at the fast-to-slow transition

Time

Que

ue le

ngth

20 packet window

30 packet window

12

vj-ll-jul06

Queue behavior with ack-per-window receiver

Time

Que

ue le

ngth

13

vj-ll-jul06

Three minor (and completely standard) variations in protocol implementation give

three wildly different average queue lengths.

I.e., the average queue lengthcontains no informationabout demand or load.

14

vj-ll-jul06

A mathematical digression ...

15

vj-ll-jul06

∫[A(t) − D(t)]

• A queue is the integral of the difference between an arrival process & a departure process.

• For packet network queues, D is usually deterministic and A is some sort of random mixture process.

16

vj-ll-jul06

The Poisson arrival process is beloved by academics

the world over

• It describes a low density, IID, uniform random collection of stuff.

• ‘Uniform random’ means the interarrival time

is distributed as e-λt.17

vj-ll-jul06

0 20 40 60 80 100

010

2030

4050

Demand (% of output capacity)

Ave

rage

Que

ue L

engt

hFor a Poisson process, queue length is a function of demand

18

vj-ll-jul06

0 5 10 15 20 25

05

1015

2025

In fact, inverse queue length is

a linear function of 1/λ

19

vj-ll-jul06

For years people haveused queue length as

a proxy for loadin network controls

• In theory this shouldn’t work at all

• In practice it sometimes sort of works because the internet protocols are robust in the face of foolishness.

20

vj-ll-jul06

Theory failures

• There can be lots of Poisson traffic in the network but not at a bottleneck.

• Poisson requires independent, random uniform traffic but at a bottleneck real traffic is highly correlated and not at all uniform.

• The correlations are intrinsic since reliability requires a sender-receiver-sender loop and traffic in a loop is never poisson.

21

vj-ll-jul06

Poisson model isn’t even representative

• Queue length is the integral of an arrival rate (with deterministic departures).

• Queue length for a Poisson process is proportional to arrival rate.

• Implies family where ∫F(t) ∝ F(t)

• Exponential (Poisson process) is the only member of this family.

22

vj-ll-jul06

• Poisson models fail because they’re memoryless but congested router queues are all about memory.

• A much better mathematical model for real queues is a random process with memory. I.e., a random walk or Brownian motion.

• Its behavior is the polar opposite of poisson (see Feller, vol.1, chap.3).

23

vj-ll-jul06

Two simulation runs of the simplest random walk

(Bernoulli trials)

0 2000 4000 6000 8000 10000

050

100

150

200

Round

Sum

0 2000 4000 6000 8000 10000

!50

050

Round

Sum

24

vj-ll-jul06

Probability density distribution of the simplest random walk

!100 !50 0 50 100

1.0

1.5

2.0

2.5

3.0

Value after 100 iterations

Pro

babi

lity

(%)

25

vj-ll-jul06

And now back to packets ...

26

vj-ll-jul06

CS profs try to instill intuition based on this picture of traffic:

Measured traffic usually looks more like:

Where does all the structure come from?27

vj-ll-jul06

• Part of the structure comes from the stability of high rate, bursty arrivals on the upstream side of the queue -- one conversation is unlikely to insert a packet into another’s burst.

• This tendency to preserve bursts is then driven by some of the non-linear dynamics of the net ...

28

vj-ll-jul06

The Internetat 50,000 feet

the Internet

Sender Receiver

packet

ack

29

The ends are just mirrors, all the dynamics result from the (usually non-linear) behavior of the net.

vj-ll-jul06

How shared links make bursts

• Picture two conversations sharing a congested gateway as two separate train tracks with one common section.

• When a blue train waits for red trains to go through the shared section, the blue trains behind it catch up (get more clumped)

• If the merge rules are efficient (service each color to exhaustion), the system clumps exponentially fast.

30

vj-ll-jul06

R1

R2

G1 G2

S1

S2

• Clumping creates a ‘horizon problem’ that tends to bias traffic managers.

• For satellite systems, the traffic time structure can interact with superframe epoch structure in unfortunate ways (e.g., ACTS hopping beams).

• Increasing delay or epoch length makes things worse (windows increase to compensate).

31

vj-ll-jul06

Suggestions

• Queue length is meaningless (but long term min can be useful).

• Try have at least a bandwidth*delay of buffer.

• Don’t let it stay full.

• Try for a ‘flow-thru’ architecture to minimize packet time-structure disturbance

32

vj-ll-jul06

Suggestions (cont.)

• Never introduce additional delay (apps will just try to fill it with packets):

‣ Let apps do their own FEC; avoid link layer Reed-Solomon and ARQ.

‣ Use smooth, simple downlink schedulers

‣ Use predictive and anticipatory uplink schedulers

33