end-to-end network performance estimation using signal complexityslides

.

What is the Signal?

• BLACK BOX: end to end (e2e) network, parts of network, data centers,network/data services, applications, etc.◦ in this paper specifically the e2e network, even more specifically availablebandwidth (AB)

• INPUT: active probing, basically trains of packets thrown at e2e paths

• OUTPUT: the same packet trains on the other side

• OBJECTIVE: to measure/understand/model the black box 02

02 myself+1 "Modeling Network Performance of End Hosts" IEICE Trans. vol.E95-D, no.7 (2012)

M.Zhanikeev -- [email protected] -- E2E Network Performance Estimation Using Signal Complexity -- http://tinyurl.com/kyutech131113 --- 2/19...

2/19

.

What is the Problem?

• it is really really hard 06 11

• multiple tools and methods, all come up withmismatched results [07]

• many design tricks, little math [08] [13]

• too much specificity, no unified/universal tool

06 M.Jain+1 "End-to-End Available Bandwidth: Dynamics..." IEEE/ACM Trans., vol.11 (2003)

11 C.Dovrolis+2 "What do packet dispersion techniques measure?" INFOCOM, vol.2 (2001)


3/19

.

The Problem: Specifics (AB)

• this paper talks specifically about available bandwidth (AB)◦ free capacity on some end-to-end path◦ between 6 and 30 hops in real networks

• it is a realtime technology!

• we cannot stress the network too much◦ example: if you want to find out how long some guy can hold his/her breath under

water, it is wrong to drown the person to find out◦ this paper will call that the Brutal Method


4/19

.

The Problem: Classification 1• obvious diagonal grouping

• IGI 07 andPathChirp 08 are the two comparison targets

◦ NOTE: so far all three are in the same cell

Low Precision High Precision

Short trains IGI, PathChirpThis Method

Long Trains iPerfPathLoad

07 N.Hu+1 "Evaluation and Characterization of AB ... Techniques" IEEE JSAC, vol.21, no.6 (2003)

08 V.Ribeiro+4 "pathChirp: Efficient Available Bandwidth Estimation for Paths" PAM Workshop (2003)


5/19

.

The Problem: Classification 2• this is where the proposed method finds its distinction• the basic idea: simple trains, complex analysis

◦ this paper uses Permutation Entropy (PE) 09

Simple Design Intricate Design

Short trains This Method IGI, PathChirp

Long Trains iPerfPathLoad

09 C.Bandt+1 "Permutation Entropy: A Natural Complexity Measure for Time Series" Physical Review Letters, Vol.88(17) (2002)


6/19

.

Things that Don't Work (1)

• this isPathChirp 08

• idea: exponentially shrinkingtrains will self-load (temporarily)

• the upper plot is the expected trend

1 2 3 45

6

7

8

Time/packet sequence

Arri

val g

ap -

Dep

artu

re g

ap

Breaking point

Dep

artu

re g

ap 12

34 5 6 7 8

Exponentialcurve

08 V.Ribeiro+4 "pathChirp: Efficient Available Bandwidth Estimation for Paths" PAM Workshop (2003)


7/19

.

Things that Don't Work (2)

• this is PathChirp inreal life

0 20 40 60 80 100Probe packet/time sequence

-0.2

0.2

0.6

1

Mixe

d n

orm

alize

d v

alue

s

back-to-back (arrival)pathchirp (arrival)

pathchirp (departure)pathchirp (diff)

01 myself "e2eprobe: source code of probing methods" https://github.com/maratishe/e2eprobe (2013)


8/19

.

Other Things that Don't Work

• any kind of groping by making small adjustment to packet space, like in IGI 07• different combinations of packet size in the same probe

• basic packet pair (just two packets) cannot measure AB, either 11

• ... good comparative study at 07

• ... source code for IGI, PathChirp and the proposal at 01


11 C.Dovrolis+2 "What do packet dispersion techniques measure?" INFOCOM, vol.2 (2001)




9/19

.

The Solution: Observations

.The most frequent pattern is .....

.

... that packets experience extreme states of being affected by thenetwork

• see the PathChirp in action a couple of slides ago

• the pattern is found for most train designs, single packets, packet pairs, etc.

.The Solution therefore is.....

.

...to analyze data as a combination of binary states (zeros and ones,basically)


10/19

.

The Solution: PE (algorithm)• arrival packet gaps

{x(i), i = 1, 2, ..

}• embed into m-dimensional space:Xi = [x(i), x(i+ d), ..., x(i+ (m− 1)d)]◦ for each i, there are multiple is◦ warping is allowed to maintain same size in all vectors◦ d is delay, the paper uses d = 1, so only neighbors -- supported by practice 02

• each Xi turned into the vector [0, 1, ...,m− 1] where value is the other, i.e.0th, 1th in Xi.

• m! permutations for each vector, each becomes a symbol• signal is then simply m-d space using in total i symbols for each permutation

• result: normalized entropy of the set of symbols◦ in this paper the best m (smallest entropy) is found using 10

02 myself+1 "Modeling Network Performance of End Hosts" IEICE Trans. vol.E95-D, no.7 (2012)

09 C.Bandt+1 "Permutation Entropy: A Natural Complexity Measure for Time Series" Physical Review Letters, Vol.88(17) (2002)

10 A.Brandmaier "Permutation Distribution Clustering and Structural Equation Model Trees" PhD Diss. (2012)


11/19

.

The Solution: Actual Methods

• static method (BC: bottleneck capacity) (r: tuning parameter = 0.8)

AB = BC− rBC(1− PE). (1)

• adaptive method: reference frame via calibration (max/min AB andPE values)

AB = ABmax −(ABmax − ABmin)(PE− PEmin)

(PEmax − PEmin). (2)


12/19

.

Experiments: Probing Network

Probing point 1

Probingtarget

labanother lab

another location in the city

Trafficdump

Probing point 2

Probing point 3

P2P1

P3


13/19

.

Experiments: Methods

• using software in 01 (IGI,PathChirp, PE)

• IGI and PathChirp are noisy,so catch when diagonal lineis crossed (simple heuristic)

• PE methods (0.8 and adaptive)

are from the previous slide --no probe design

40 50 60 70 80 90 100Sending rate (Mbps)

40

50

60

70

80

90

100

Rece

ivin

g ra

te (M

bps)

pathchirpigi

Detection point



14/19

.

Experiments: One Run• real probing in the wild

• prefer to run all the methods roughly at thesame time

• PE and Brute AB are based onback-to-back probe

Pick randompsize,

probesize

Run the probing

client

Storedata on

server side

Probingstream

To the next run

1 Back-to-back probe2 Pathchirp probe3 IGI probe * 5 gaps Feedback

var psize; // random [100, 1000]var probesize; // random [50, 350]

// open TCP client to remote IP and portvar client = new Client( rip, rport);

// run back-to-back probe// … and wait for RX throughputrunBackToBackTX( psize, probesize); var thru = client->waitForStatus();

// pathchirp probevar low = 0.5 * thru;var high = 5 * thru;var alpha = pickAlpha( low, high);runPathChirpTX( psize, low, high, alpha);

// IGI probevar low = 0.5 * thru;var high = 1.5 * thru;var step = ( high - low) / 5;for ( var rate = low; rate <= high; rate += step) {

runIgiTX( psize, probesize, rate);}


15/19

.

Results (1)• result: 0.8 is better on average, but adaptive is the best!• performance varies with network distance

30 50 70 90Brute AB

30

50

70

90

Estim

ated

AB pathchirp

igiigi

adaptive0.8 0.8

P120 40 60 80

Brute AB

20

40

60

80

100

Estim

ated

AB

pathchirpigi

adaptive

0.8 0.8

P210 30 50 70 90

Brute AB

1030507090

Estim

ated

AB

pathchirpigi

igiadaptive

adaptive0.80.8

P3

56 70 84 98AB

0102030

40

Erro

r

pathchirpigi

igi

adaptive0.8 0.8

30 50 70 90AB

0

20

4050

Erro

r

pathchirpigi

igi

adaptive

adaptive

0.8

0.8

30 50 70 90AB

40

60

Erro

r

pathchirp

pathchirp

igi

igi

adaptive adaptive

0.8

0.8


16/19

.

Results (2) : CCF Analysis

P vs

Brute AB

Brute AB vs

Brute PE

Brute AB vs

Pathchirp AB

Brute AB vs

IGI AB

-0.6

-0.2

0.2

0.6

CC

F

-0.3

-0.58

0.40.53


17/19

.

Wrapup

• PE helps by

1. using dumb probes with simple designs2. reducing analysis to complexity of binary response

• the method can be used for any metric, not just AB◦ jitter◦ tomography (sensing of topology)◦ load or utilization of a system◦ etc....

• will cover all these in future work ...


18/19

.

That’s all, thank you ...


19/19

.

[01] myself (2013)e2eprobe: source code of probing methodshttps://github.com/maratishe/e2eprobe

[02] myself+1 (2012)Modeling Network Performance of End HostsIEICE Trans. vol.E95-D, no.7

[03] 1+myself+1 (2006)Rate-Based and Gap-Based Available Bandwidth Estimation...Springer LNCS, vol.4238

[04] 1+myself+1 (2006)ABshoot: A Reliable and Efficient Scheme for End-to-End AB...IEEE TENCON

[05] 1+M.Zhanikeev (2009)Active Network Measurement: Theory, Methods, and ToolsITU Japan


19/19

.

[06] M.Jain+1 (2003)End-to-End Available Bandwidth: Dynamics...IEEE/ACM Trans., vol.11

[07] N.Hu+1 (2003)Evaluation and Characterization of AB ... TechniquesIEEE JSAC, vol.21, no.6

[08] V.Ribeiro+4 (2003)pathChirp: Efficient Available Bandwidth Estimation for PathsPAM Workshop

[09] C.Bandt+1 (2002)Permutation Entropy: A Natural Complexity Measure for Time SeriesPhysical Review Letters, Vol.88(17)

[10] A.Brandmaier (2012)Permutation Distribution Clustering and Structural Equation Model TreesPhD Diss.


19/19

.

[11] C.Dovrolis+2 (2001)What do packet dispersion techniques measure?INFOCOM, vol.2

[13] J.Strauss+2 (20s03)A Measurement Study of Available Bandwidth Estimation Tools3rd ACM SIGCOMM


19/19

end-to-end network performance estimation using signal complexityslides

Technology

signal complexity

parts of network

bandwidth abe2e network

packet trains

packets experience network

pathchirp short trains

natural complexity measure

e2e paths output