copa: practical delay-based congestion control...copa gets higher throughput cubic flows are not...

Copa: Practical Delay-Based Congestion Control

Venkat Arun and Hari Balakrishnan

MIT, CSAILweb.mit.edu/copa

The Internet is more challenging than ever(Why are we still talking about congestion control in 2018?)

The Internet is more challenging than ever

Greater bandwidth ⇒Higher bandwidth-delay product

Lower tolerance for non-congestive loss

Greater flow-churn

(Why are we still talking about congestion control in 2018?)

The Internet is more challenging than ever

Greater bandwidth ⇒

Large flows (e.g. video streaming) co-exist with short-flows

Higher bandwidth-delay product

Lower tolerance for non-congestive loss

Greater flow-churn

(Why are we still talking about congestion control in 2018?)

The Internet is more challenging than ever

Greater bandwidth ⇒

Large flows (e.g. video streaming) co-exist with short-flows

Wireless links with variable bandwidths are commonplace

Higher bandwidth-delay product

Lower tolerance for non-congestive loss

Greater flow-churn

(Why are we still talking about congestion control in 2018?)

The Internet is more challenging than ever

Greater bandwidth ⇒

Large flows (e.g. video streaming) co-exist with short-flows

Wireless links with variable bandwidths are commonplace

Higher bandwidth-delay product

Lower tolerance for non-congestive loss

Greater flow-churn

Simultaneously, users are more sensitive to performance!

(Why are we still talking about congestion control in 2018?)

Loss-based schemes have long-standing problems

• Buffer-filling

• Vulnerable to non-congestive loss

• Loss is a coarse signalWorsens with increasing bandwidth

Delay-based congestion control?

Benefits Challenges

Delay-based congestion control?

Maintain low delay Not competitive with buffer-filling schemes

Benefits Challenges

Delay-based congestion control?

Maintain low delay

Robust to misleading loss

Not competitive with buffer-filling schemes

Delay can mislead too!

Benefits Challenges

Delay-based congestion control?

Maintain low delay

Robust to misleading loss

Not competitive with buffer-filling schemes

Delay can mislead too!

Benefits Challenges

Rich signal

Delay-based congestion control?

Maintain low delay

Robust to misleading loss

Not competitive with buffer-filling schemes

Delay can mislead too!

Finding true minimum RTTis hard

Benefits Challenges

Rich signal

Delay-based congestion control?

Not competitive with buffer-filling schemes

Delay is noisy too!

Finding true minimum RTTis hard

Our SolutionChallenges

Delay-based congestion control?

If buffer-fillers are present, give up on low delay

Not competitive with buffer-filling schemes

Delay is noisy too!

Finding true minimum RTTis hard

Our SolutionChallenges

Delay-based congestion control?

If buffer-fillers are present, give up on low delay

MIN: a more robust statistic for queuing delay

Not competitive with buffer-filling schemes

Delay is noisy too!

Finding true minimum RTTis hard

Our SolutionChallenges

Delay-based congestion control?

If buffer-fillers are present, give up on low delay

MIN: a more robust statistic for queuing delay

Not competitive with buffer-filling schemes

Delay is noisy too!

Finding true minimum RTTis hard

Our SolutionChallenges

Empty queues periodically

Basic Goals

Avoid congestion collapse+

Efficient and Fair allocation of bandwidth+

Low delay

Target rate = r( = )*+,-

Adjustable Parameterdefault = 0.5

Queuing delay

Target Rate ≡ Nash Equilibrium

"#$%$#&' = log #,-# − /'log(12)Assuming Poisson arrivals (more details in paper)

Selfishly optimize for:

Target Rate ≡ Nash Equilibrium

"#$%$#&' = log #,-# − /'log(12)Assuming Poisson arrivals (more details in paper)

Selfishly optimize for:

Unique and Efficient

Computing the Target Rate

Estimating queuing delay from RTT

Time

RTT

True minimum RTT

RTT

RTTminQueuing delay

Estimating queuing delay from RTT

Time

RTT

True minimum RTT

RTT

RTTminRTT – RTTmin = queuing delay

Queuing delay

Estimating queuing delay from RTT

Time

RTT

True minimum RTT

RTTmin

New flow starts here

Wrong!

Estimating queuing delay from RTT

Time

RTT

True minimum RTT

True minimum RTT = RTTmin for new flows!

A “noisy” cellular link: Stanford to AWSDe

lay

expe

rienc

ed b

y in

divi

dual

pac

kets

(ms)

Time (s)

Decoupling queuing delay from other delay variation

Time

Que

ue le

ngth Measured RTT

• Wireless links• Cross traffic• ACK compression• Bursty transmission• …

Decoupling queuing delay from other delay variation

Time

Que

ue le

ngth Measured RTT

• Wireless links• Cross traffic• ACK compression• Bursty transmission• …

Positive Additive Noise!

Take Min over last 0.5 RTT of samples

A “noisy” cellular link: Stanford to AWS

Using the MIN delay estimator improves throughput from0.5 Mbits/s to 3.9 Mbits/s

Dela

y ex

perie

nced

by

indi

vidu

al p

acke

ts

Time (s)

Attaining the Target

Calculate target rate = '( = )*+,

If current rate < r.: additively increase by 0* pkts/RTTElse: additively decrease by 0* pkts/RTT

The Copa Algorithm

Calculate target rate = '( = )*+,

If current rate < r.: additively increase by 0* pkts/RTTElse: additively decrease by 0* pkts/RTT

The Copa AlgorithmVelocity forfaster convergence

Calculate target rate = '( = )*+,

If current rate < r.: additively increase by 0* pkts/RTTElse: additively decrease by 0* pkts/RTT

22

24

26

28

0 0.2 0.4 0.6 0.8 120

22

24

26

28

30

Cw

nd (p

kts)

RTT

stan

ding

(ms)

Time (s)

CwndRTTstanding

The Copa Algorithm

RTTCwnd

Cwnd

(pkt

s)

RTT

(ms)

Time (s)

Velocity forfaster convergence

Steady-State Dynamics of CopaQ

ueue

Leng

th (i

n pk

ts)

Time

Equilibrium queue length

2.5 $%&

1 $%&

00 RTT 0.5 RTT. 1.5 RTT 2.5 RTT 3 RTT 4 RTT

Period: 5 RTTsAmplitude: 2.5 $%&

Steady-State Dynamics of CopaQ

ueue

Leng

th (i

n pk

ts)

Time

Equilibrium queue length

2.5 $%&

1 $%&

0

0.5 RTT

window

Switch point

Queue length corresponding toRTTstanding at the switch point

0 RTT 0.5 RTT. 1.5 RTT 2.5 RTT 3 RTT 4 RTT

Period: 5 RTTsAmplitude: 2.5 $%&

Steady-State Dynamics of CopaQ

ueue

Leng

th (i

n pk

ts)

Time

Equilibrium queue length

2.5 $%&

1 $%&

0

0.5 RTT

window

Switch point

Queue length corresponding toRTTstanding at the switch point

0 RTT 0.5 RTT. 1.5 RTT 2.5 RTT 3 RTT 4 RTT

Period: 5 RTTsAmplitude: 2.5 $%&

Steady-State Dynamics of CopaQ

ueue

Leng

th (i

n pk

ts)

Time

Equilibrium queue length

2.5 $%&

1 $%&

0

0.5 RTT

window

Switch point

Queue length corresponding toRTTstanding at the switch point

0 RTT 0.5 RTT. 1.5 RTT 2.5 RTT 3 RTT 4 RTT

Period: 5 RTTsAmplitude: 2.5 $%&

Steady-State Dynamics of CopaQ

ueue

Leng

th (i

n pk

ts)

Time

Equilibrium queue length

2.5 $%&

1 $%&

0

0.5 RTT

window

Switch point

0.5 RTTwindow

Queue length corresponding toRTTstanding at the switch point

Switch point

0 RTT 0.5 RTT. 1.5 RTT 2.5 RTT 3 RTT 4 RTT

Period: 5 RTTsAmplitude: 2.5 $%&

Detect buffer-filling TCP

Estimate true minimum RTT

Queue empties every 5 RTTs! ⇒

TCP-Competitiveness

Mode switching for TCP competitiveness

0

5

10

15

20

25

0 10 20 30 40 50 600

5

10

15

20

Thro

ughp

ut (M

bits

/s)

Que

uing

Del

ay (m

s)

Time (s)

Delay sensitive(! = 0.5)

TCP CubicCopa

Mode switching for TCP competitiveness

0

5

10

15

20

25

0 10 20 30 40 50 600

5

10

15

20

Thro

ughp

ut (M

bits

/s)

Que

uing

Del

ay (m

s)

Time (s)

Delay sensitive(! = 0.5)

TCP CubicCopa

Low Delay Low Throughput Low Delay

Mode switching for TCP competitiveness

0

5

10

15

20

25

0 10 20 30 40 50 600

5

10

15

20

Thro

ughp

ut (M

bits

/s)

Que

uing

Del

ay (m

s)

Time (s)

0

5

10

15

20

25

0 10 20 30 40 50 600

5

10

15

20

Thro

ughp

ut (M

bits

/s)

Que

uing

Del

ay (m

s)

Time (s)

Delay sensitive(! = 0.5)

TCP Competitive(AIMD on !&')

TCP CubicCopa

Low Delay Low Throughput

High ThroughputHigh Delay

Low Delay

High Delay

TCP CubicCopa

Mode switching for TCP competitiveness

0

5

10

15

20

25

0 10 20 30 40 50 600

5

10

15

20

Thro

ughp

ut (M

bits

/s)

Que

uing

Del

ay (m

s)

Time (s)

0

5

10

15

20

25

0 10 20 30 40 50 600

5

10

15

20

Thro

ughp

ut (M

bits

/s)

Que

uing

Del

ay (m

s)

Time (s)

Delay sensitive(! = 0.5)

TCP Competitive(AIMD on !&')

TCP CubicCopa

Low Delay Low Throughput

High ThroughputHigh Delay

Low Delay

High Delay

TCP CubicCopa

Mode switching for TCP competitiveness

0

5

10

15

20

25

0 10 20 30 40 50 600

5

10

15

20

Thro

ughp

ut (M

bits

/s)

Que

uing

Del

ay (m

s)

Time (s)

TCP Cubic

Copa

Low Delay High Throughput Low Delay

When queue doesn’t empty once every 5 RTTs, switch to TCP Competitive mode!

Best of both worlds!

0

0.5

1

1.5

2

2.5

Vegas Reno Cubic Copa PCC BBR

Avg.

(Tpt

/ Id

eal t

pt)

Scheme under test's throughputCubic's throughput

Copa gets higher throughput without hurting TCP Cubic!

0

0.5

1

1.5

2

2.5

Vegas Reno Cubic Copa PCC BBR

Avg.

(Tpt

/ Id

eal t

pt)

Scheme under test's throughputCubic's throughput

Copa gets higher throughput without hurting TCP Cubic!

0

0.5

1

1.5

2

2.5

Vegas Reno Cubic Copa PCC BBR

Avg.

(Tpt

/ Id

eal t

pt)

Scheme under test's throughputCubic's throughput

Copa gets higher throughput without hurting TCP Cubic!

Copa gets higher throughput Cubic flows are not hurt

Limitations

• Cannot ignore low frequency noise

• Queues don’t empty periodically if:• Propagation delay is much smaller than queuing delay• Flows with very different propagation delays share a bottleneck queue

• Needs precise RTT measurements

Consistent Performance on Real Paths

0

0.2

0.4

0.6

0.8

1

16326412825651210242048

BBR

Copa

CubicLEDBAT

PCC

Sprout

RemyVegasVerus

Vivace latency

Vivace lossVivace LTE

Avg

. Nor

mal

ized

Thr

ough

put

Avg. Queuing Delay (ms)

Cellular Networks

0

0.2

0.4

0.6

0.8

1

124816

BBRCopaCubic

LEDBAT

PCC

Sprout

Remy

VegasVerus Vivacelatency

Vivace lossVivace LTE

Avg.

Nor

mal

ized

Thr

ough

put

Avg. Queuing Delay (ms)

Wired Networks

Better⊳

Better⊳

Satellite link: High BDP, high loss

010203040

800 110100

CopaRemyPCC

Cubic Vegas

BBR

Thro

ughp

ut (M

bit/s

)

Median Queuing Delay (ms)

Better⊳

Fairness during flow-churn

1

10

1 3 5 7 9 11 13 15 17

1

10

1 3 5 7 9 11 13 15 171

10

1 3 5 7 9 11 13 15 17

1

10

1 3 5 7 9 11 13 15 17

Copa

BBR

Cubic

PCC

Thro

ughp

ut (M

bit/

s)Av

g±

Std.

Dev

.

Time (s) Time (s)

Fairness during flow-churn

1

10

1 3 5 7 9 11 13 15 17

1

10

1 3 5 7 9 11 13 15 171

10

1 3 5 7 9 11 13 15 17

1

10

1 3 5 7 9 11 13 15 17

Copa

BBR

Cubic

PCC

Thro

ughp

ut (M

bit/

s)Av

g±

Std.

Dev

.

Time (s) Time (s)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

CopaCu

bicBBRPC

C

Jain index

Better⊳

Summary

NetworkEstimate queuing delay

and compute target

!

A practical delay-based congestion control algorithmhttps://web.mit.edu/copa

Summary

NetworkEstimate queuing delay

and compute target

!

Outer control loopMode switching andCompetitive mode

A practical delay-based congestion control algorithmhttps://web.mit.edu/copa

Summary

NetworkEstimate queuing delay

and compute target

Move toward target (AIAD-variant)

!

Outer control loopMode switching andCompetitive mode

A practical delay-based congestion control algorithmhttps://web.mit.edu/copa

Rate/cwnd