application-level network performance / measurement tools

University of Illinois at ChicagoElectronic Visualization Laboratory (EVL)

Application-Level Network Performance / Measurement Tools

Jason Leigh, Oliver Yu,

Alan Verlo, Linda Winkler


Tele-Immersion is the synthesis of Virtual Reality, video conferencing, and advanced computation


Example of a Data Intensive Tele-Immersion Application

• TIDE: Tele-Immersive Data Explorer

• Collaborative Large Scale Data Visualization

• In collaboration with National Center for Data Mining

• General framework for collaborative visualization of massive data-sets

• Current data-set is ozone data from NOAA


Example of a Data Intensive Tele-Immersion Application

• CIBRView: Collaborative Image Based Rendering Viewer

• Cosmology Hydrodynamic code by Julian Borrill, LBNL/NERSC shows theoretical condensation of diffuse matter into string-like formations during early stages of universe evolution

• Accesses volume data:512x256x256x 256 frames ~ 40Gig data-sets

• Generates image slices that are distributed to collaborating clients

• Sent about 500, 1M slices/filesfrom Chicago to Japan using parallelTCP.

• It was also the application over whichwe tested DiffServ.


BW Latency Sensitive

Jitter Sensitive

Reliability Reqrmnt

Burstiness Packet Size

Protocol

Avatar (virtual representation)

Low Y Y Low Constant Small RTP / UDP

Audio conference

Med Y Y Low Constant Med RTP / UDP

Video conference

High Y Y Low Constant Large RTP / UDP

Realtime state updates

Low Y Y Med Short bursts

Small FEC

Non-realtime state updates

Low N N High App dependent

Small TCP

Small bulk data

(image files)

Med N N High Medium bursts

Large TCP/CPTCP

Large bulk data(raw data sets)

High N N High Very long bursts

Large PTCP / RBUDP

Streaming bulk data

(high quality audio/video/bitmap/polygons)

High Y N High Long bursts

Large SRBUDP / AFEC

Characteristics of Tele-Immersion Flows


Motivation for Application-level networking tools

• Bandwidth is becoming increasingly available. • Networking QoS is still under research and difficult to deploy and

use. It is not as easy as “flipping a switch.”• Network QoS is not only about bandwidth, it’s about latency and

jitter.• Applications today are not ready to use the extra bandwidth even

if available.• Application developers have to be increasingly network savvy in

order to be able to convert application requirements to networking services.

• Need to make advanced networking easier for the average application developer.

• Need to provide a higher level application framework to keep pace with network advances.


Advanced Data Transport Techniques for Tele-Immersion

• Maybe we can compensate for latency:– Reliable Low-Latency Data Transfer for Tele-

Immersion

• Even if you had QoS could you really take advantage of it?– High Throughput Techniques for Tele-Immersion


Reliable Low-Latency Data Transfer for Tele-Immersion

• In Tele-Immersion it is desirable to be able to transmit state information with minimum latency and jitter while preserving reliability

• Rather than use TCP which uses acknowledgments to obtain reliability, try UDP augmented with error correction codes:Forward Error Correction


`Latency of transmitting 100 packets underUDP, TCP, FEC/UDP with 3:1 redundancy from EVL (Chicago) to

SARA (Amsterdam)

0

50

100

150

200

250

300

350

400

0 500 1000 1500 2000 2500

Packet size in bytes

1-w

ay la

ten

cy in

ms

UDP

TCP

FEC over UDP

FEC greatest benefit is in small packets.

Larger packets impose greater overhead.

As redundancy decreases FEC approaches UDP.

goal


Jitter for UDP, TCP and FEC over UDPMoving average (over 20 successive data points) of deviations of Short Term Latency (also over 20

successive data points)

0

2

4

6

8

10

12

14

1 5 9

13

17

21

25

29

33

37

41

45

49

53

57

61

65

69

73

77

Jit

ter

UDP

TCP

FEC/UDP

G o a l


Packet Loss over UDP vs FEC/UDP between Chicago & Amsterdam

50Mbps UDP or FEC

+50Mbps UDP congestion

Packet Loss

UDP 1.90%

FEC 0.05%

UDP with congestion 17.40%

FEC with congestion 4.15%


High Throughput Techniques for Tele-Immersion

• In Tele-Immersion it is desirable to share data files as rapidly as possible

• Even if you had QoS, you couldn’t take advantage of it

• Long Fat Network problem: an ftp session will max out at 3.5Mbps over a 100Mbps link between Chicago and Amsterdam (and Switzerland)

• 2 Techniques:– Parallel TCP Socket Striping– Reliable Blast UDP


Parallel Sockets : 64K Window SizeAmsterdam (SARA) to Chicago (EVL)

Plot of Average Achievable Bandwidth vs # of Paralle TCP Sockets Used to Deliver a 50M File from Amsterdam to Chicago over 100Mbps Link

0

10

20

30

40

50

60

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

# of Sockets

Band

wid

th (M

bps)

Found it difficult to achieve more than50Mbps on a 100Mbps link.Have been able to achieve 80Mbpson rare occasions.


Reliable Blast UDP (RBUDP)

• RBUDP - An old idea that may be useful now that networking bandwidth is increasing

• Use UDP for bulk data transmission rather than TCP

• If bandwidth can be guaranteed by QoS –reliability will be high- chances of errors will be few


RBUDP Performance(EVL to SARA)

Average throughput vs transmission unit size(size before an ACK is required) over 100Mbps link between EVL and SARA.

01020

30405060

708090

1 2 3 4 6 8 10 15 20 22 23 25 30 40 60 80 100 150 200 400

Unit Size (MBytes)

Aver

age

Tho

ught

put

(Mbp

s)

Realtime Throughput Maximum UDP throughput


RBUDP Performance(EVL to CERN)

Streaming RBUDP from CERN to EVL over 45Mbps link

0

5

10

15

20

25

30

35

Unit Size (Mbytes)

Th

rou

gh

pu

t (M

bp

s)

RUDP sending at 33Mbps

Maximum throughput achieved by UDP(33Mbps)


DiffServ results over a testbed between EVL and Argonne National Lab

25Mbps80Mbps

ANL

EVL

42Mbps 42Mbps

100Mbps 100Mbps

100Mbps 100Mbps

fore

back

x

x

x x

Cisco7507

DiffServ Bandwidth

0

5

10

15

20

25

1 10

19

28

37

46

55

64

73

82

91

100

109

118

127

136

145

154

163

Time (s)

Ban

dw

idth

(M

bp

s)

DiffServ Latency

0

50

100

150

200

250

300

1 10

19

28

37

46

55

64

73

82

91

100

109

118

127

136

145

154

163

Time (s)

1 w

ay L

ate

ncy (

ms

)

DiffServ Packet Loss

0200400600800

1000120014001600

1 9 17 25 33 41 49 57 65 73 81 89 97 105

113

121

129

137

145

153

161

Time (s)

Pac

ke

t L

os

s (

pac

ke

ts/s

)

Bandwidth recovery good

Latency recovery not good

Packet loss double

+ background + DiffServ

150ms 1-way latency


Collaborative Coordination Experiments betweenChicago and Singapore

• Tightly coordinated collaborative interaction task between 2 remote users

• 200ms RTT is the threshold where performance begins to suffer• 200ms RTT with 0 jitter is same as 10ms RTT with 7ms jitter• So DiffServ is not suited for Tele-Immersion


• G2 is C++ toolkit for building Tele-Immersive applications with special emphasis on networking

• G2 is the Grid’s main tele-immersion library• Networking:

– UDP, TCP, Multicast, HTTP.– UDP reflector and multicast bridge.– TCP reflector.– Remote procedure calls.– 32 and 64bit Remote file I/O.– Parallel 32 & 64 bit TCP socket striping for high throughput data

delivery.– FEC.– Client/Server distributed shared memory persistent database.– Threading, Mutual Exclusion.– Built-in Instrumentation of networking services.


• Tools for higher level application development:– Audio streaming– Articulated Avatars– VR navigation– VR menus– Speech recognition with IBM ViaVoice– Collaborative application shell to jumpstart development– Network visualization tools


QoS Internet Monitoring ToolQoSIMoto

• Provides real time viewing of CAVERNsoft data streams

• Visualizes bandwidth, latency, jitter of multiple network flows

• Accepts Netlogger compatible format


These techniques are needed more than ever in STAR LIGHT

• Being able to dedicate lambdas will ease our latency and jitter problems but we still can’t beat the speed of light.

• Lambda switching requires more intelligence at the edges to perform traffic shaping, bandwidth management, error correction.

• Need to provide high level API for applications to select Lambdas and define which application flow will go over the lambda.

• Long Fat Network problems do not go away with GMPLS.• Need to provide a higher level application programmer’s

framework to keep pace with network advances. We need more than just sockets( ) API.


What new things can we do withSTAR LIGHT?

• Streaming uncompressed high resolution stereoscopic 3D movies (1024x768x24bitsx30fps) ~ 1.05 Gbps

• Imagine magnifying this to multi-tiled displays for ultra high resolution displays such as CAVEs and Active Murals



Distributed Tera-snap:• Perform a tera-mining correlation between

distributed databases and generate a visual overview

• 1 PC can absorb ~500Mbps => min 4.6 hours to perform a tera-snap

• 20 PCs can absorb data and produce image composites in a min of 13 mins using 10Gbps



Digital Continuums: Distributed collaboratories with linked tiled, tele-immersive and mobile displays enhanced with cluster computing and data mining services


Thanks

• Thanks SARA (Amsterdam), CERN (Switzerland), IHPC (Singapore) for graciously participating in these network experiments

• For more info:– www.evl.uic.edu/cavern– [email protected]

application-level network performance / measurement tools

Documents