![Page 1: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/1.jpg)
Fair queueing and congestion controlJim Roberts (France Telecom)
Joint work with Jordan Augé
Workshop on Congestion ControlHamilton Institute, Sept 2005
![Page 2: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/2.jpg)
Fairness and congestion control
s fair sharing: an objective as old as congestion controlQcf. RFC 970, Nagle, 1985
Qnon-reliance on user cooperation
Qpainless introduction of new transport protocols
Qimplicit service differentiation
s fair queueing is scalable and feasibleQaccounting for the stochastics of traffic
Qa small number of flows to be scheduled
Qindependent of link speed
s performance evaluation of congestion controlQmust account for realistic traffic mix
Qimpact of buffer size, TCP version, scheduling algorithm
![Page 3: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/3.jpg)
Flow level characterization of Internet traffic
s traffic is composed of flowsQan instance of some application
Q(same identifier, minimum packet spacing)
s flows are "streaming" or "elastic"Qstreaming SLS = "conserve the signal"
Qelastic SLS = "transfer as fast as possible"
inter-packet < T silence > T
start end
streaming elastic
![Page 4: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/4.jpg)
Characteristics of flows
s arrival processQPoisson session arrivals: succession of flows and think times
s size/durationQheavy tailed, correlation
flowarrivals
start of session
end ofsession
think times
![Page 5: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/5.jpg)
Characteristics of flows
s arrival processQPoisson session arrivals: succession of flows and think times
s size/durationQheavy tailed, correlation
s flow peak rateQstreaming: rate codec
Qelastic: rate exogenous limits (access link,...)
rate
duration
rate
duration
![Page 6: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/6.jpg)
Three link operating regimes
needscheduling
excellent for elastic,some streaming loss
needoverload control
low throughput,significant loss
FIFOsufficient
negligible loss and delay
overallrate
"transparent" "congested""elastic"
flows
![Page 7: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/7.jpg)
Performance of fair sharing without rate limit (ie, all flows bottlenecked)
s a fluid simulation:QPoisson flow arrivals
Qno exogenous peak rate limit flows are all bottlenecked
Qload = 0.5 (arrival rate x size / capacity)
high
low
averagerate
start end
20 seconds
linkrate
incoming flows
![Page 8: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/8.jpg)
The process of flows in progress depends on link load
load 0.5
high
low
10
0
20
30flows in progress
![Page 9: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/9.jpg)
The process of flows in progress depends on link load
10
0
20
30flows in progress
load 0.6
high
low
![Page 10: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/10.jpg)
The process of flows in progress depends on link load
10
0
20
30flows in progress
load 0.7
high
low
![Page 11: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/11.jpg)
The process of flows in progress depends on link load
10
0
20
30flows in progress
load 0.8
high
low
![Page 12: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/12.jpg)
The process of flows in progress depends on link load
10
0
20
30flows in progress
load 0.9
high
low
![Page 13: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/13.jpg)
Insensitivity of processor sharing: a miracle of queuing theory !s link sharing behaves like M/M/1
Qassuming only Poisson session arrivals
s if flows are bottlenecked, E [flows in progress] = Qi.e., average 9 for 0.9, but as 1
s but, in practice, < 0.5 and E [flows in progress] = O(104) !
0 .2 .4 .6 .8 load,
E [flows in progress]
0
5
10
15
20
![Page 14: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/14.jpg)
Trace data
s an Abilene link (Indianapolis-Clevelend) – from NLANRQOC 48, utilization 16 %
Qflow rates (10 Kb/s, 10 Mb/s)
Q~7000 flows in progress at any time
10 sec
2.5 Gb/s
>2.5 Mb/s
< 250 Kb/s
![Page 15: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/15.jpg)
Most flows are non-bottlenecked
s each flow emits packets rarely
s little queueing at low loadsQFIFO is adequate
Qperformance like a modulated M/G/1
s at higher loads, a mix of bottlenecked and non-bottlenecked flows...
~5 µs2.5 Gb/s
~7000 flows
~1 ms15 Mb/s
![Page 16: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/16.jpg)
Fair queueing is scalable and feasible
s fair queueing deals only with flows having packets in queueQ<100 bottlenecked flows (at load < 90%)
QO(100) packets from non-bottlenecked flows (at load < 90%)
s scalable since number does not increase with link rateQdepends just on bottlenecked/non-bottlenecked mix
s feasible since max number is ~500 (at load < 90%)Qdemonstration by trace simulations and analysis (Sigmetrics 2005)
s can use any FQ algorithmQDRR, Self-clocked FQ,...
Qor even just RR ?
![Page 17: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/17.jpg)
Typical flow mix
s many non-bottlenecked flows (~104)Qrate limited by access links, etc.
s a small number of bottlenecked flows (0, 1, 2,...)QPr [ i flows] ~ i with the relative load of bottlenecked flows
s exampleQ 50% background traffic
–ie, E[flow arrival rate] x E[flow size] / capacity = 0.5
Q0, 1, 2 or 4 bottlenecked TCP flows–eg, at overall load = 0.6, Pr [ 5 flows] 0.003
![Page 18: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/18.jpg)
Simulation set up (ns2)
s one 50 Mbps bottleneckQRTT = 100ms
s 25 Mbps background trafficQPoisson flows: 1 Mbps peak rate
Qor Poisson packets (for simplicity)
s 1, 2 or 4 permanent high rate flowsQTCP Reno or HSTCP
s buffer sizeQ20, 100 or 625 packets (625 = b/w x RTT)
s schedulingQFIFO, drop tail
QFQ, drop from front of longest queue
![Page 19: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/19.jpg)
Results:- 1 bottlenecked flow,- Poisson flow background
![Page 20: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/20.jpg)
FIFO + Reno20 packets 625 packets
1000
1
100s 100s
cwnd(pkts)
utilization
0
0
![Page 21: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/21.jpg)
FIFO + Reno1000
1
100s 100s
cwnd(pkts)
utilization
20 packets 100 packets
0
0
Severe throughput loss with small buffer: - realizes only 40% of available capacity
![Page 22: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/22.jpg)
FIFO + 100 packet buffer1000
1
100s 100s
cwnd(pkts)
utilization
0
0
Reno HSTCP
HSTCP brings gain in utilization, higher loss for background flows
![Page 23: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/23.jpg)
Reno + 20 packet buffer1000
1
100s 100s
cwnd(pkts)
utilization
0
0
FIFO FQ
FQ avoids background flow loss,little impact on bottlenecked flow
![Page 24: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/24.jpg)
Results:- 2 bottlenecked flows,- Poisson packets background
![Page 25: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/25.jpg)
FIFO + Reno + Reno1000
1
100s 100s
cwnd(pkts)
utilization
0
0
20 packets 625 packets
Approximate fairness with Reno
![Page 26: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/26.jpg)
FIFO + HSTCP + HSTCP1000
1
100s 100s
cwnd(pkts)
utilization
0
0
20 packets 625 packets
![Page 27: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/27.jpg)
FIFO + HSTCP + Reno1000
1
100s 100s
cwnd(pkts)
utilization
0
0
20 packets 625 packets
HSTCP is very unfair
![Page 28: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/28.jpg)
Reno + HSTCP + 20 packet buffer1000
1
100s 100s
cwnd(pkts)
utilization
0
0
FIFO FQ
![Page 29: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/29.jpg)
Reno + HSTCP + 625 packet buffer1000
1
100s 100s
cwnd(pkts)
utilization
0
0
FIFO FQ
Fair queueing is effective (though HSTCP gains more throughput)
![Page 30: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/30.jpg)
Results:- 4 bottlenecked flows,- Poisson packet background
![Page 31: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/31.jpg)
All Reno + 20 packet buffer1000
1
100s 100s
cwnd(pkts)
utilization
0
0
1 flow 4 flows
Improved utilization with 4 bottlenecked flows,approximate fairness
![Page 32: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/32.jpg)
All Reno + 625 packet buffer1000
1
100s 100s
cwnd(pkts)
utilization
0
0
1 flow 4 flows
Approximate fairness
![Page 33: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/33.jpg)
All HSTCP + 625 packet buffer1000
1
100s 100s
cwnd(pkts)
utilization
0
0
1 flow 4 flows
Poor fairness, loss of throughput
![Page 34: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/34.jpg)
All HSTCP + 625 packet buffer1000
1
100s 100s
cwnd(pkts)
utilization
0
0
FIFO FQ
Fair queueing restores fairness, preserves throughput
![Page 35: Fair queueing and congestion control Jim Roberts (France Telecom) Joint work with Jordan Augé Workshop on Congestion Control Hamilton Institute, Sept 2005](https://reader030.vdocuments.mx/reader030/viewer/2022032800/56649d235503460f949fa2bb/html5/thumbnails/35.jpg)
Conclusions
s there is a typical traffic mixQsmall number of bottlenecked flows (0, 1, 2,...)
Qlarge number of non-bottlenecked flows
s fair queueing is feasibleQO(100) flows to schedule for any link rate
s results for 1 bottlenecked flow + 50% backgroundQsevere throughput loss for small buffer
QFQ avoids loss and delay for background packets
s results for 2 or 4 bottlenecked flows + 50% backgroundQReno approximately fair
QHSTCP very unfair, loss of utilization
QFQ ensures fairness for any transport protocol
s alternative transport protocols ?