CapProbe: An Efficient and CapProbe: An Efficient and Accurate Capacity Estimation Accurate Capacity Estimation

TechniqueTechnique

Sigcomm 2004Sigcomm 2004

Rohit Kapoor**, Ling-Jyh Chen*, Li Lao*, Rohit Kapoor**, Ling-Jyh Chen*, Li Lao*, M.Y. Sanadidi*, Mario Gerla*M.Y. Sanadidi*, Mario Gerla*

** Qualcomm Corp R&D** Qualcomm Corp R&D*UCLA Computer Science Department*UCLA Computer Science Department

The Capacity Estimation ProblemThe Capacity Estimation Problem

Estimate minimum link capacity on Estimate minimum link capacity on an Internet path, as seen at the IP an Internet path, as seen at the IP levellevel

Design GoalsDesign Goals End-to-endEnd-to-end: assume no help from : assume no help from

routersrouters InexpensiveInexpensive: Minimal additional traffic : Minimal additional traffic

and processing at end nodesand processing at end nodes FastFast: converges to capacity fast : converges to capacity fast

enough for the applicationenough for the application

ApplicationsApplications TCP parameters optimizationTCP parameters optimization Adaptive multimedia streamingAdaptive multimedia streaming Overlay network structuringOverlay network structuring Wireless link monitoring and mobility Wireless link monitoring and mobility

detectiondetection

T3

T2 T3

T3

T1

T3

Narrowest Link

20Mbps 10Mbps 5Mbps 10Mbps 20Mbps 8Mbps

Packet Pair Dispersion and CapacityPacket Pair Dispersion and Capacity

Ideal Packet DispersionIdeal Packet Dispersion FIFO routers, no cross-trafficFIFO routers, no cross-traffic

Capacity = (Packet Size) / (Dispersion)

Expansion of DispersionExpansion of Dispersion Cross-traffic (CT) serviced between PP packetsCross-traffic (CT) serviced between PP packets Second packet queues due to Cross Traffic (CT )=> Second packet queues due to Cross Traffic (CT )=>

expansion of dispersion =>expansion of dispersion =>UnderUnder-estimation-estimation

Compression of DispersionCompression of Dispersion First packet queueing => First packet queueing =>

compressed dispersion => compressed dispersion => OverOver-estimation-estimation

Previous WorkPrevious Work Dovrolis’ WorkDovrolis’ Work

Analyzed under/over estimation of capacityAnalyzed under/over estimation of capacity Designed PathrateDesigned Pathrate

• First send packet pairsFirst send packet pairs• If multimodal, send packet trainsIf multimodal, send packet trains• Identifies modes to distinguish ADR Identifies modes to distinguish ADR

(Asymptotic Dispersion Rate), PNCM (Asymptotic Dispersion Rate), PNCM (Post Narrow Capacity Mode) and (Post Narrow Capacity Mode) and Capacity Modes Capacity Modes

Previously proposed techniques have relied Previously proposed techniques have relied either on either on dispersiondispersion or or delaydelay

Key ObservationKey Observation First packet queues (First packet queues (postpost bottle-neck) bottle-neck)

CompressionCompressionCapacity over-estimationCapacity over-estimation

Second packet queues (Second packet queues (prepre or or postpost b-neck) b-neck)• ExpansionExpansion• Under-estimationUnder-estimation

Both Both over and under estimationover and under estimation are the result are the result of probe packets experiencing queuing delayof probe packets experiencing queuing delay• E-to-E delay min when queuing delay = 0E-to-E delay min when queuing delay = 0

CapProbe ApproachCapProbe Approach

Filter PP results that have queuing time > 0, ie Filter PP results that have queuing time > 0, ie not the minimum E to E delaynot the minimum E to E delay

Dispersion of PP with “Dispersion of PP with “minimum delay summinimum delay sum” ” (of the two packets in the pair) reflects (of the two packets in the pair) reflects capacitycapacity

CapProbe combines both CapProbe combines both dispersiondispersion and e2e and e2e transit transit delaydelay information information

Techniques for Convergence DetectionTechniques for Convergence Detection Consider set of packet pair probes 1…nConsider set of packet pair probes 1…n

If If min(d1) + min(d2) ≠ min(d1+d2),min(d1) + min(d2) ≠ min(d1+d2), dispersion dispersion obtained from min delay sum may be obtained from min delay sum may be distorteddistorted• Above condition increases correct Above condition increases correct

detection probability to that of a single detection probability to that of a single packet (as opposed to packet pair)packet (as opposed to packet pair)

If above minimum delay sum condition is not If above minimum delay sum condition is not satisfied in a runsatisfied in a run New run, with packet size of probesNew run, with packet size of probes

• IncreasedIncreased if bandwidth estimated varied a if bandwidth estimated varied a lot across probeslot across probes

• DecreasedDecreased if bandwidth estimated varied if bandwidth estimated varied little across probeslittle across probes

SimulationsSimulations

6-hop path: capacities {10, 7.5, 5.5, 6-hop path: capacities {10, 7.5, 5.5, 44, 6, 8} Mbps , 6, 8} Mbps PP pkt size = PP pkt size = 200200 bytes, CT pkt size= 1000 bytes bytes, CT pkt size= 1000 bytes Persistent Persistent TCPTCP Cross-Traffic Cross-Traffic

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 1.6 3.2 4.8 6.4 8Bandwidth Estimate (Mbps)

Fre

qu

ency

1Mbps2Mbps4Mbps

Over-Estimation

Cross Traffic Rate

Bandwidth Estimate Frequency

0

0.001

0.002

0.003

0.004

0.005

0.006

0.007

0.008

0.009

0.01

0 1.6 3.2 4.8 6.4 8Bandwidth Estimate (Mbps)

Min

Del

ay S

um

s (s

ec)

1Mbps2Mbps4Mbps

Cross Traffic Rate

Minimum Delay Sums

SimulationsSimulations

PP pkt size = CT pkt size = PP pkt size = CT pkt size = 500500 bytes bytes Non-Persistent Non-Persistent TCPTCP Cross-Traffic Cross-Traffic

0

0.0021

0.0042

0.0063

0 1.6 3.2 4.8 6.4 8Bandwidth Estimate (Mbps)

Min

Del

ay S

um

(se

c)

1Mbps

3Mbps

Minimum Delay Sums

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 1.6 3.2 4.8 6.4 8Bandwidth Estimate (Mbps)

Fre

qu

ency

1Mbps

3Mbps

Under-Estimation

Bandwidth Estimate Frequency

SimulationsSimulations

Non-Persistent Non-Persistent UDPUDP CBR Cross-Traffic CBR Cross-Traffic

Case where CapProbe does not workCase where CapProbe does not work UDP (non-responsive), extremely intensiveUDP (non-responsive), extremely intensive No correct samples are obtainedNo correct samples are obtained

0

0.002

0.004

0.006

0.008

0.01

0.012

0.014

0 1.6 3.2 4.8 6.4 8Bandwidth Estimate (Mbps)

Min

Del

ay S

um

s (s

ec)

1Mbps2Mbps3Mbps4Mbps

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 1.6 3.2 4.8 6.4 8Bandwidth Estimate (Mbps)

Fre

qu

ency

1Mbps2Mbps3Mbps4Mbps

Minimum Delay SumsBandwidth Estimate

Frequency

CapProbeCapProbe Sufficient requirementSufficient requirement

At least one PPAt least one PP sample where both packets sample where both packets experience experience nono CT induced queuing CT induced queuing delaydelay..

How realistic is this requirement?How realistic is this requirement? Internet is reactive (mostly TCP): high Internet is reactive (mostly TCP): high

chance of some probe packets not being chance of some probe packets not being queuedqueued

To validate, we performed extensive To validate, we performed extensive experimentsexperiments• Only cases where such samples are not Only cases where such samples are not

obtained is when cross-traffic is UDP and obtained is when cross-traffic is UDP and very intensive (typically >75% load)very intensive (typically >75% load)

Probability of Obtaining SampleProbability of Obtaining Sample

Assuming PP samples arrive in a Assuming PP samples arrive in a PoissonPoisson manner manner Product of probabilitiesProduct of probabilities

No queue in front of first packet: p(0) = 1 – λ/μ No queue in front of first packet: p(0) = 1 – λ/μ No CT packets enter between the two packets No CT packets enter between the two packets

(worst case)(worst case)• Only dependent on arrival processOnly dependent on arrival process

p = p(0) * e- λL/μ = (1 – λ/μ) * e- λL/μ p = p(0) * e- λL/μ = (1 – λ/μ) * e- λL/μ Analysis also for deterministic and Pareto cross-Analysis also for deterministic and Pareto cross-

traffictraffic

Link

No Cross Traffic Packets

First Packet

No Queue

Second Packet

Probability of Obtaining Sample (cont)Probability of Obtaining Sample (cont)

Avg number of samples required to obtain an

unqueued PP for a single link; Poisson cross-traffic

Avg number of samples required to obtain an

unqueued PP for a single link; LRD cross-traffic

ExperimentsExperiments Simulations, Internet, Internet2 (Abilene), Simulations, Internet, Internet2 (Abilene),

WirelessWireless TCP (responsive), CBR (non-responsive), LRD TCP (responsive), CBR (non-responsive), LRD

(Pareto) cross-traffic(Pareto) cross-traffic Wireless technologies tested were Bluetooth, Wireless technologies tested were Bluetooth,

IEEE 802.11, 1xRTTIEEE 802.11, 1xRTT Path-persistent, non-persistent cross-trafficPath-persistent, non-persistent cross-traffic

Internet MeasurementsInternet Measurements Each experiment: 500 PP at 0.5s intervalsEach experiment: 500 PP at 0.5s intervals 100 experiments for each {Internet path, nature 100 experiments for each {Internet path, nature

of CT, narrow link capacity}of CT, narrow link capacity}

To

UCLA-2 UCLA-3 UA NTNU

Time Capacity Time Capacity Time Capacity Time Capacity

CapProbe

0’03 5.5 0’01 96 0’02 98 0’07 97

0’03 5.6 0’01 97 0’04 79 0’07 97

0’03 5.5 0’02 97 0’17 83 0’22 97

Pathrate

6’10 5.6 0’16 98 5’19 86 0’29 97

6’14 5.4 0’16 98 5’20 88 0’25 97

6’5 5.7 0’16 98 5’18 133 0’25 97

Pathchar

21’12 4.0 22’49 18 3 hr 34 3 hr 34

20’43 3.9 27’41 18 3 hr 34 3 hr 35

21.18 4.0 29’47 18 3 hr 30 3 hr 35

IssuesIssues CapProbe may be implemented either in the CapProbe may be implemented either in the

kernel or user modekernel or user mode Kernel mode more accurate, particularly over Kernel mode more accurate, particularly over

high-speed linkshigh-speed links One-way or round-trip estimationOne-way or round-trip estimation

One-way One-way • Requires cooperation from receiverRequires cooperation from receiver• Can be used to estimate forward/reverse Can be used to estimate forward/reverse

linklink Active vs passiveActive vs passive Heavy cross-trafficHeavy cross-traffic

Difficulty in correct estimationDifficulty in correct estimation

SummarySummary

CapProbe is CapProbe is accurate, fast, and accurate, fast, and inexpensive, across a wide range of inexpensive, across a wide range of scenariosscenarios

Potential applications in overlay Potential applications in overlay structuring, and in case of fast structuring, and in case of fast changing wireless link speedschanging wireless link speeds

High-speed dispersion High-speed dispersion measurements needs more measurements needs more investigationinvestigation