masaki hirabaru internet architecture group gl meeting march 19, 2004 high performance data transfer...
TRANSCRIPT
Masaki Hirabaru<[email protected]>
Internet Architecture Group
GL MeetingMarch 19, 2004
High Performance Data transfer onHigh Bandwidth-Delay Product Networks
VLBI (Very Long Baseline Interferometry)
(CRL鹿島宇宙電波応用Gホームページから )
Motivations• MIT Haystack – CRL Kashima e-VLBI Experiment
on August 27, 2003 to measure UT1-UTC in 24 hours– 41.54 GB CRL => MIT 107 Mbps (~50 mins)
41.54 GB MIT => CRL 44.6 Mbps (~120 mins)
– RTT ~220 ms, UDP throughput 300-400 MbpsHowever TCP ~6-8 Mbps (per session, tuned)
– BBFTP with 5 x 10 TCP sessions to gain performance
• HUT – CRL Kashima Gigabit VLBI Experiment
- RTT ~325 ms, UDP throughput ~70 MbpsHowever TCP ~2 Mbps (as is), ~10 Mbps (tuned)
- Netants (5 TCP sessions with ftp stream restart extension)
They need high-speed / real-time / reliable / long-haul high-performance, huge data transfer.
Purpose
• Measure, analyze and improve end-to-end performance in high bandwidth-delay product networks– to support for networked science applications– to help operations in finding a bottleneck– to evaluate advanced transport protocols
(e.g. Tsunami, SABUL, HSTCP, FAST, XCP, ikob)
• Improve TCP under easier conditions
Assumptions• Packet Switching Network
– Shared, Best-Effort
• End-to-End Principle– No hard state
Contents• Advanced TCP evaluation on TransPAC / Internet2• Advanced TCP evaluation in laboratory• Research Topics in 2004
KwangjuBusan
2.5G
Fukuoka
Korea
2.5G SONET
KORENTaegu
Daejon
10G
0.6G1Gx2
1Gx2
QGPOP
Seoul XP
Genkai XP
Kitakyushu
Tokyo XP
Kashima
0.1G
Fukuoka Japan
250km
1,000km2.5G
TransPAC
9,000km
4,000km
Los Angeles
Cicago
New York
MIT Haystack
HUT
10G
1G
APII/JGN
Abilene
0.1GHelsinki
2.4G
Stockholm
0.6G
2.4G
2.4G
GEANT
Nordunet
funetKoganei
1G
7,000km
Indianapolis
I2 Venue1G
10G
100km
server (general)
server (e-VLBI)
Abilene Observatory: servers at each NOC
CMM: common measurement
machines
Network Diagram for TransPAC/I2 Measurement(Oct. 2003)
1G x2
sender
receiver
Mark5Linux 2.4.7 (RH 7.1)P3 1.3GHzMemory 256MBGbE SK-9843
PE1650Linux 2.4.22 (RH 9)Xeon 1.4GHzMemory 1GBGbE Intel Pro/1000 XT
Iperf UDP ~900Mbps (no loss)
TransPAC/I2 #2: High Speed (60 mins)
Evaluating Advanced TCPs
• new Reno (Linux TCP, web100 version)– Ack: w=w+1/w, Loss: w=w-1/2*w
• HighSpeed TCP (included in web100)– Ack: w=w+a(w)/w, Loss: w=w-1/b(w)*w
• FAST TCP (binary, provided from Caltech)– w=1/2*(w_old*baseRTT/avgRTT+α+w_current)
• Limited Slow-Start (included in web 100)
Note: Differences in sender side only
Path
ReceiverSender
Backbone
B1 <= B2 & B1 <= B3
Access Access
B1B2
B3
a) w/o bottleneckqueue
ReceiverSender
Backbone
B1 > B2 || B1 > B3
Access Access
B1 B2 B3
b) w/ bottleneck (congestion)queue
bottleneck
Test in a laboratory – with bottleneck
PacketSphere
ReceiverSender
L2SW(12GCF)
Bandwidth 800Mbps Buffer 256KBDelay 88 msLoss 0
GbE/SX
GbE/TGbE/T
PE 2650 PE 1650
• #4: Reno => Reno
• #5: High Speed TCP => Reno
• #6: FAST TCP => Reno
2*BDP = 16MB
#4, 5: Data obtained on sender #6: Data obtained on receiver
Laboratory #4,#5,#6: 800M bottleneck
Reno
HighSpeed
FAST
(default setting)
Laboratory #5: High Speed (Limiting)
Window Size(16MB)
Rate Control
Cwnd Clamp
270 us every 10 packetsWith limited slow-start (1000)
(95%)With limited slow-start (100)
With limited slow-start (1000)
Laboratory #6: FAST
(alpha=17, beta=18,
gamma=2)
Research topics in 2004
Internet
Bottleneck
ReceiverSender Router
Bottleneck
ACK (100 ms delay)
100 x 1KB = 100KB ~ 1 ms at Gbps
100M1G
1G
100M
Throughput
Window size
•Parameter Auto-tuning (w/ packet pair/train)
•TCP Analysis WEB Tool with web100
•Router queue length
•RED on the bottleneck
•Signal from the bottleneck
•Mobile TCP (Mobile router support)
•Multi-Homing TCP (ID+Locator)
•I2 / piPEs collaboration (bwctl)
•APII / CMM collaboration with Korea
•Real-time Gbps e-VLBI experiment between CRL Kashima and MIT Haystack