INCITE –Edge-based Traffic Processing

for High-Performance Networks

R. Baraniuk, E. Knightly, R. Nowak, R. Riedi Rice University

L. Cottrell, J. Navratil, W. MathewsSLAC

W. Feng, M. GardnerLANL

web site: incite.rice.edu

INCITE Project – Rice, SLAC, LANL 2 incite.rice.edu

INCITE Project• InterNet Control and Inference from The Edge

on-line tools to characterize and map host and network performance as a function of time, space, application, protocol, and service

INCITE Project – Rice, SLAC, LANL 3 incite.rice.edu

INCITE Thrusts and Tools

Thrust 1: Multiscale traffic analysis and modeling techniques

o wavelet, multifractal, connection-level models

Thrust 2: Inference and control algorithms for network paths, links, and routers

o end-to-end path probing and modelingo network tomography and topology discoveryo advanced high-speed protocols

Thrust 3: Data collection tools

o active measurement infrastructureo passive application-layer measurement

INCITE Project – Rice, SLAC, LANL 4 incite.rice.edu

pathChirp• Goal

– estimate instantaneous available bandwidth (ABW) on an end-to-end network link

• Basic probing paradigm– stream packets at some rate

no queuing delay rate<ABW queuing delay builds up

rate>ABW• Until now: tradeoff

– high accuracy has required high volume probing (inefficient)

• Unique to pathChirp – variable rate probe packet train

(exponentially spaced chirp)– 10x more efficient than

competing techniques

INCITE Project – Rice, SLAC, LANL 5 incite.rice.edu

Network TomographyFrom end-to-endmeasurements…

… infer internal topology and delay/loss characteristics

INCITE Project – Rice, SLAC, LANL 6 incite.rice.edu

TCP - Low Priority

• TCP alone 745.5 Kb/s

• TCP plus 739.5 Kb/sTCP-LP 109.5 Kb/s

• TCP-LP is invisible to TCP

• Goal– utilize excess bandwidth in a

non-intrusive fashion • Methodology

– sender-side modification of TCP: delay-based approach

• Applications– bulk data transfers– available bandwidth monitoring– P2P file sharing

• High-speed TCP-LP– TCP-LP + HSTCP– implementation

Linux-2.4.22-web100

– experiments Stanford - Ann Arbor Stanford - Gainesville

R 1 R 2

TC P-L P

TC P

C = 1 .5 M b/s

cro s s - t ra f f ic

INCITE Project – Rice, SLAC, LANL 7 incite.rice.edu

Advanced TCP stacks• Standard TCP (Reno) has problems on today’s long-

distance high-speed networks (e.g. trans ocean/continent > hundreds of Mbits/s)

• Advanced TCP stacks (e.g. FAST, High-speed, TCP-LP …) and new rate based UDP transports address this issue

• We have evaluated many (~10) new implementations for throughput, stability, fairness, ease of use etc.

• BaBar (HENP) tier A sites (e.g. SLAC, IN2P3 (Lyon Fr) and FZK (Karlsruhe)) now starting to use chosen TCP stack for production transfer of Monte Carlo data to SLAC– Easier to use than multi-stream TCP, only optimize

one parameter (window size)

INCITE Project – Rice, SLAC, LANL 8 incite.rice.edu

Changes in network topology (BGP) can result in dramatic changes in performance

Snapshot of traceroute summary table

Samples of traceroute trees generated from the table

ABwE measurement one/minute for 24 hours Thu 9 Oct 9:00am to Fri 10 Oct 9:01am

Drop in performance(From original path: SLAC-CENIC-Caltech to SLAC-Esnet-LosNettos (100Mbps) -Caltech )

Back to original path

Changes detected by IEPM-Iperf and AbWE

Esnet-LosNettos segment in the path(100 Mbits/s)

Hour

Rem

ote

host

Dynamic BW capacity (DBC)

Cross-traffic (XT)

Available BW = (DBC-XT)

Mbit

s/s

Note:1. Caltech misrouted via Los-Nettos 100Mbps commercial net 14:00-17:002. ESnet/GEANT working on routes from 2:00 to 14:00

Los-Nettos (100Mbps)

INCITE Project – Rice, SLAC, LANL 9 incite.rice.edu

Crossing the Application/Network Divide

Application

TCP

IP

Data Link

Network

Send dataover network

Segmentation

Fragmentation

Flow & Congestion Control

Checksums

::

• Implications to the application?• Insights for high- performance network

protocols?

Network monitors focus here.

INCITE Project – Rice, SLAC, LANL 10 incite.rice.edu

TICKET and MAGNET+MUSETICKET: Traffic Information-Collecting Kernel with Exact Timing

MAGNeT: Monitor for Application-Generated Network TrafficMUSE: MAGNET User-Space Environment

Application

TCP

IP

Data Link

Network

MAGNET

Send dataover network

Segmentation

Fragmentation

Flow & Congestion Control

Checksums

MUSE

TICKET:tcpdump++

::

For more information, go to www.lanl.gov/radiant/pubs.html

INCITE Project – Rice, SLAC, LANL 11 incite.rice.edu

MAGNeT MAGNET Monitoring Apparatus for General kerNel-Event Tracing (at nanoscale granularity)

• Why not extend monitoring to kernel events in general? Software Oscilloscope for Cluster and Grids – Debugging

e.g., IdentifiedLinux OS bug in the scheduler for SMPs. Can be used to deploy, debug, and monitor the DOE

UltraNet (UltraScienceNet), e.g., dynamic provisioning.– Performance Optimization

Improved performance of 10GigE adapters by 300%. Can improve end-to-end performance of DOE UltraNet.

– Monitoring Grid Applications Integrated MAGNET with SciDAC’s PERC TAU and

SciDAC’s PERC SvPablo/Autopilot.*– Adaptive Resource-Aware Applications

• SciDAC Deployment: PERC, Supernova Science Ctr, Transit Network Fabric + Terascale Supernova Initiative + Fusion Energy (emerging), and Earth Systems Grid II (emerging).

* For more information, see M. Gardner, W. Deng, T. Markham, C. Mendes, W. Feng, and D. Reed, “A High-Fidelity Software Oscilloscope for Globus,” GlobusWorld 2004, Jan. 2004.