rice university, spin group spin.rice.edu multiscale traffic processing techniques for network...

Rice University, SPiN Group spin.rice.edu

Multiscale Traffic Processing Techniques for Network Inference and Control

R. Baraniuk R. Nowak E. Knightly R. Riedi

X. Wang V. Ribeiro S. Sarvotham Y. Tsang

NMS PI meeting Baltimore April 2002

Signal Processing in Networking


Chirp Probing

Effort 1


Objective: Reduced complexity, multiscale

link models with known accuracy

Innovative Ideas Multifractal analysis Multiplicative modeling Multiscale queuing Chirps for probing

Impact Congestion control

Workload balancing at servers Dynamical streaming Pricing on connection basis


Chirp Probe Cross-Traffic Inference


New Ideas

Probing multiple hopsChirp-Sandwiches to probe routers

“down the path”IP-tunneling

Monitor packets at intermediate point of pathVerification for Chirping and Sandwiches

Network calculus (max-plus algebra)

Probing buffer at core router Passive inference


Tech Transfer

- Stanford (SLAC) Chirps in PingER as freeware for monitoring- C+ code for sending Chirp Probes- 10 Msec precision- Visualization tools for link properties

IP tunneling- Infrastructure at Rice ready

- CAIDA (chirping as a monitoring tool)

- UC Riverside (Demystify Self-similarity)

- Sprint Labs (queue-sizes at core routers)


Connection-level Analysis and

Modeling of Network Traffic

Effort 2


Aggregate Traffic at small scales

• Trace:– Time stamped headers

• Large scales– Gaussian – LRD (high variability)

• Small scales– Non-Gaussian – Positive process– Burstiness

Objective :– Origins of bursts

Auckland Gateway (2000)Aggregate Bytes per time

Gaussian: 1%Real traffic: 3%

Kurtosis - Gaussian : 3 - Real traffic: 5.8

Mean

99%

0 0.5 1 1.5 2 2.5 3

x 105

0

50

100

150

200

250

300

350

400

450histogram

Gaussian: 1%Real traffic: 3%

0 2000 4000 60000

0.5

1

1.5

2

2.5

3x 10

5

time (1 unit=500ms)nu

mbe

r of

byt

es

Mean

99%


• ON/OFF model – Superposition of sources– Connection level model

• Explains large scale variability: – LRD, Gaussian– Cause: Costumers – Heavy tailed file sizes !!

Bursts in the ON/OFF framework

• Small scale bursts:– Non-Gaussianity– Conspiracy of sources ??– Flash crowds ??

(dramatic increase of active sources)


Non-Gaussianity: A Conspiracy?

• The number of active connections is close to Gaussian; provides no indication of bursts in the load

• Indication for:- No conspiracy of sources

- No flash crowds

Load: Bytes per 500 ms

Number of active connections

Mean

99%

Mean

99%


Non-Gaussianity: a case study

Typical bursty arrival(500 ms time slot)

Typical Gaussian arrival(500 ms time slot)

Histogram of load offered in same time bin per connection:Considerable balanced “field” of connections

10 Kb

Histogram of load offered in same time bin per connection:One connection dominates

150 Kb

15 Kb


Non-Gaussianity and Dominance

Systematic study: time series separation• For each bin of 500 ms:

remove packets of the ONE strongest connection• Leaves “Gaussian” residual traffic

Overall traffic Residual traffic1 Strongest connection

= +Mean

99%


Separation on Connection Level

Definition:• Alpha connections:

Peak rate > mean arrival rate + 1 std dev

• Beta connections: Residual traffic

• C+ analyzer (order of sec)• Separation is more efficient (less accurate)

using multi-scale analysis


Beta Traffic Component

• Constitutes main load• Governs LRD properties of overall traffic• Is Gaussian at sufficient utilization (Kurtosis = 3)

• Is well matched by ON/OFF model

Beta trafficVariance time plot

Mean

99%

Traffic percentile: 1%

Recall: beta = non-dominatingRecall: beta = non-dominating

0 0.5 1 1.5 2 2.5

x 105

0

50

100

150

200

250

300

350

400

Histogram of Beta traffic

Beta Traffic Kurtosis 3.4

Number of connections = ON/OFF

Mean

99%


Origins of Alpha Traffic

• Mostly TCP (flow & congestion controlled).• HTTP, SMTP, NNTP, etc.• Alpha connections tend to have the same source-

destination IP addresses.• Any large volume connection with the same e2e IP

addresses as an alpha connection is also alpha.

Systematic cause of alpha:Large file transfers over high bandwidth e2e paths.


Simple Connection Taxonomy

This is the only systematic reason

Bursts arise from largetransfers over fast links.

.RTTcwnd

bandwidth .RTTcwnd

bandwidth


Cwnd or RTT?

Correlation coefficient=0.68

RTT has strong influence on bandwidth and dominance.

103

104

105

106

10-1

100

101

102

peak-rate (Bps)

1/R

TT

(1/

s)

Correlation coefficient=0.01

103

104

105

106

102

103

104

105

peak-rate (Bps)

cwnd

(B

)

Colorado State University trace, 300,000 packets

cwnd 1/RTT

ratepeak cwnd

1/RTTratepeak

Beta Alpha Beta Alpha


Examples of Alpha/Beta Connections

0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4

-200

0

200

400

600

800

1000

1200

1400

one beta connection

packet arrival time (second)

pack

et s

ize (

byte

s)

forward direction

reverse direction

41.3 41.35 41.4 41.45

-200

0

200

400

600

800

1000

1200

1400

one alpha connection (96078, 196, 80, 59486)

packet arrival time (second)

pack

et s

ize

(byt

es)

forward direction

reverse direction

Alpha connections burst because of short round trip time, not large TCP window

Modeling of Alpha Traffic• ON/OFF model revisited:

High variability in connection rates (RTTs)

Low rate = beta High rate = alpha

fractional Gaussian noise stable Levy noise

+

=

+

+

=

Modeling of Alpha Traffic• ON/OFF model revisited:

High variability in connection rates (RTTs)

Low rate = beta High rate = alpha

fractional Gaussian noise stable Levy noise

0 2000 4000 60000

0.5

1

1.5

2

x 105

time (1 unit=500ms)

num

ber

of b

ytes

0 2000 4000 60000

0.5

1

1.5

2

2.5

3x 10

5

time (1 unit=500ms)

num

ber

of b

ytes


Impact: Simulation

• Simulation: ns topology to include alpha links

Simple: equal bandwidth Realistic: heterogeneous end-to-end bandwidth

• Congestion control• Design and management


Impact: Performance

• Beta Traffic rules the small Queues• Alpha Traffic causes the large Queue-sizes

(despite small Window Size)

All Alpha Packets Queue-size with Alpha Peaks


Multiscale Nature of Traffic

• LRD– Large time scales – approx. Gaussian – Client behavior– Bandwidth over Buffer

• Multifractal– Small time scale– Mixture Gaussian - Stable– Network topology– Control at Connection level

Structure: Multiplicative Additive

packetscheduling

sessionlifetime

networkmanagement

round-triptime

< 1 msec msec-sec minutes hours


Ongoing Work

• Versatile efficient additive/multiplicative models (fast synthesis, meaning of model parameters)

• Impact of Alpha traffic on Performance• Goal: Obtain performance parameters

directly from network and user specifications

• Monitoring tool for network load (freeware) using chirps (Caida, SLAC)

• Verification/”internal” measurements using IP-tunneling


Summary• Alpha traffic (peak rate > burst threshold)

– Few connections. Responsible for bursts– Origin: Large transfer over high bandwidth paths– High bandwidth from short RTT

• Beta traffic (residual): – Main load. Responsible for LRD– Origin: Crowd with limited bandwidth– Gaussian at sufficient utilization

• Connection level analysis: parameter estimation drastically simplifies using multi-scale analysis

• Statistical modeling: requires novel multi-scale models (mixtures of additive and multiplicative trees)

rice university, spin group spin.rice.edu multiscale traffic processing techniques for network...

Documents

alpha connections

real traffic

kurtosis gaussian

aggregate traffic

number of connections

large volume connection

connection leveldefinition

small scale bursts