Network Emulation for Researching Long-Fat Networks, Delay Tolerant

Networks and Grid Environments

Presented by:Eric Coe

Computer Systems Research Department, M1–102The Aerospace Corporation

www.aero.orgecoe@aero.org(310) 336-1911

February 9, 2005

2

Outline

• Aerolab: Aerospace Testbed based on EMULAB– What it is, how it works, why we need it

• Define the following and motivate why network emulation is appropriate

– Long Fat Network (aka High-Speed Bandits)– Delay or Disruption Tolerant Networks– Grid Environments

• Demo: So how easy is it?• Future Directions

3

What is Aerolab?

Time- and Space-Shared Network Emulation System • Construction of Complex Topologies• Emulation of Link Conditions—Delay, Loss and Bandwidth

(dynamic)• Boot Multiple Operating Systems—FreeBSD, Linux, and U of

Utah’s OS-Kit (real-time Linux, Windows on the Way)• Runs on Commodity Hardware—PCs and Cisco Switch• Scales to Hundreds of Physical Nodes and supports Multiple

Logical Nodes per Physical Nodes

Easy-To-Use Front-End• Automates Configuration of OS, Topology and Links• Supports both ns2 style scripting and Web-based GUI

Originally developed by University of Utah under a National Science Foundation Grant

4

Aerolab Architecture

10x5x1Gbps

10x1Gbps

Control LAN

ExperimentLAN

ExperimentalNodes

ProgrammableSwitch

ExperimentStorage

“Users”Server

“Boss”Server

Web/DB/SNMP Switch

Mgmt

I nternet

10x Serial

PowerController

Power

Serial

TerminalServer

Ca talys t 6 5 00 S ER IES

5

Utah’s Emulab

programmableswitch

physicalnodes

controlnodes

6

Why Network Emulation (Utah)

“We evaluated our system on five nodes.”-job talk from university with 300-node cluster

“We evaluated our Web proxy design with 10 clients on 100Mbit ethernet.”

“Simulation results indicate ...” “Memory and CPU demands on the individual nodes

were not measured, but we believe will be modest.” “The authors ignore interrupt handling overhead in their

evaluation, which likely dominates all other costs.” “Resource control remains an open problem.”

7

Why Network Emulation (Aerospace)

• “Different projects need to evaluate/operate different networks”

• “Network set-up and running cables is costly and open to configuration errors (HW/SW)”

• “Aerolab should hide the networking “magic” so that researchers can experiment at the application or protocol level”

• “In theory simulation and the real-world are the same but in practice they are different”

• “Repeatability, Repeatability, Repeatability and archiving. Old demos can be run w/ little overhead.”

8

Why Network Emulation (more..)

• “You have to know the right people to get access to the cluster or real network.”

• “The cluster or real network is hard to use.”• “<Experimental network X> runs FreeBSD 2.2.x.”• “October’s schedule for <experimental network Y> is…”• “<Experimental network Z> is tunneled through the

Internet”

9

Aerolab’s Value to Aerospace

Enhances Networking Capabilities• Large Scale Experiments• Complex Topologies• Hardware and Software “In The Loop”

Automates Laboratory Operations• Reduces Labor and Complexity• Improves Repeatability and Allows “Archiving” Experiments• Facilitates Resource Sharing

Emulab is not just the latest cool thing, it’s the right way to run a networking lab.

10

Outline

• Aerolab: Aerospace Testbed based on EMULAB– What it is, how it works, why we need it

• Define the following and motivate why network emulation is appropriate

– Long Fat Network (aka High-Speed Bandits)– Delay or Disruption Tolerant Networks– Grid Environments

• Demo: So how easy is it?• Future Directions• Potential Collaboration

11

Long Fat Networks (LFN)

What they are:• Characterized as networks with large bandwidth-delay products• Typically gigabit speeds and have delays larger than 70 msec• Examples: networks of compute-clusters; satellite networks;

transcontinental or transoceanic high-speed links.• Network where standard TCP has poor performance

12

Long Fat Networks (LFN)

What an Aerolab like system provides:• Install custom kernel images that contain various transport

protocols (HS-TCP, BIC, CUBIC, WEB100, XCP)• Total control of the networking environment

• Run what-if scenario’s

• Guarantee no other traffic on the network or only “known” traffic

• Packet loss can be attribute to “WHO”.

• Dynamic Link Characteristics (satellite)• Varying bandwidth

• Varying error rates, probability of packet loss

• Repeatability: When results from this week do not match last weeks. .

13

Delay Tolerant Networks (DTN)

What they are:• Characterized as networks with no contemporaneous end to end

path• Occur due to intermittent connectivity which is caused by

• large propagation delays

• nodes going up and down frequently

• with highly mobile nodes in sparse network

• Networks where standard TCP has no performance due to the lack of an end to end path

14

Delay Tolerant Networks (DTN)

15

Delay Tolerant Networks (DTN)

How we classify:• Application basis

• Interplanetary Network (networking the solar system)

• Sensor networks

– Static nodes (due to nodes choosing their on and off periods

– Mobile nodes (mobility process dictates links presence or absence)– Hybrid case

• Link types• Deterministic: Contacts can be scheduled

• Probabilistic: Contacts happen based on some distribution

• Dynamic: Contacts happen in a purely opportunistic way

16

Delay Tolerant Networks (DTN)

What an Aerolab like system provides:• Dynamic link conditions for delay, bandwidth, and even link

availability • Ability to run Bundling Protocol on top of a topology and over

different topologies• Ability to evaluate routing protocols across the variety of networks

discussed earlier• A common facility for other researchers to conduct experiments

Avoids everyone having their own “version/vision” of what constitutes a DTN

17

Grid Environments

What they are:• Networks of loosely coupled computational resources• Heterogeneous by nature in both capacity and availability• Basic grid services and applications must inhabit a dynamic,

heterogeneous, distributed environment• Issue: How to quantitatively evaluate grid services and

applications in such an environment?

18

Quantitative Evaluation of Grids

What an Aerolab-like system could support for grid research:• Playback mechanism for network behavior, cross traffic• Injecting errors for testing fault tolerance, robustness• Testing autonomic computing methods (AC

control cycle: monitor, analyze, plan, execute)• Remove a link or halt processes on a host

• Evaluate response of autonomic control system

• Testing information or resource discovery on larger networks• Quantitatively evaluate scalability in larger topologies• Quantitative evaluation wrt topology structure and connectedness• Tools needed to provide these capabilities in a general way

19

USC

Aerospace

ISI

APSimScript

Globus

MDS-2

ATMD ATMD

overlay

APSimHLA/RTI

APSimHLA/RTIAPSim

HLA/RTI

start

discover

register

overlayrequest

overlayconfig

XBoneManager

Example: Evaluation of Grid Application Performance

DARPA Active Networks Project Exploit application topology to compute lower-bound timestamp in distributed simulations Globus used to manage Xbone and start HLA-compliant simulation Very hard to run large test cases

HLA Time Mgmt done“in the network” by overlay of active time mgmt daemons

20

Example: Evaluation of Grid Application Performance

Solution: Emulate larger topologies on Utah EmuLab Set of quasi-random, tree topologies generated

4, 8, 16, 32, 64 end-hosts9, 15, 22, 29, 34 interior service hosts

Tree topologies constrained to have average 2.5 degree of connectedness 550 Lower Bound Timestamp (DRN) calculations done for each topology

Example: topology w/ 32 end-hosts

21

Optimistic Mesh Generation on the Grid

Nave, Chrisochoides, Lee, Deerfield, International Meshing Roundtable, Sept. 2004

Optimistic Delauney Mesh GenerationOptimism controlled by number of cavity expansion msgs – affects BW demand, latencyAlready run on clusters and TeraGrid

Work in Progress: run on AeroLab to do repeatable parametric tests controlling available bandwidth

22

Aerolab

#1. Create a topology#2. Do something useful

Demo: So how easy is it?

23

Future Directions

• Different OSes, namely Solaris, real-time Linux and Windows

• Incorporate Hardware emulators (IXP,SPIRNET)• Incorporate passive network sniffers (layer 1) • Wireless nodes, possible mobile campus nodes• Merge with the Aerospace cluster (more nodes)• Integrate 10 Gigabit capability• Federate with other testbeds

• More nodes...more nodes...more nodes

24

Aerolab

Thank You!

Questions?