15th June 2016, Wednesday University of Washington, Seattle, USA
Mohit P. Tahiliani, Dharmendra Kumar Mishra, Pranav Vankar
Wireless Information Networking Group,
National Institute of Technology Karnataka, Surathkal, India
8th Annual Workshop on ns-3 (WNS3 2016)
TCP Evaluation Suite for ns-3
Outline of the presentation
❑ Motivation
❑ Existing implementations
❑ Design and implementation of tcp-eval in ns-3
❑ User interaction with ns-3 tcp-eval
❑ Comparing TCP extensions in ns-3
❑ Results and discussions
❑ Validation issues
❑ Conclusions and the next goals
15th June 2016, Wednesday University of Washington, Seattle, USA
Motivation
❑ Congestion control algorithms continue to evolve . . .
❑ . . . and so do TCP extensions!
❑ Problem: not feasible to evaluate every TCP extension exhaustively
❑ Potential solution:
- derive some initial results and study the behaviour
- consider the promising ones for thorough evaluation
What is TCP Evaluation Suite?
- a set of well-defined, standard test cases to compare TCP extensions
- initially proposed by Transport Modeling Research Group (TMRG)
- modified by Internet Congestion Control Research Group (ICCRG)
- widely used today for evaluating new TCP extensions
15th June 2016, Wednesday University of Washington, Seattle, USA3
Existing implementations
❑ Wang, G., Y. Xia, and D. Harrison. “An NS2 TCP evaluation tool.”
draftirtf-tmrg-ns2-tcp-tool, IETF Internet Draft (expired) (2007).
- Two versions of code.
- Version 2 source: https://sourceforge.net/projects/tcpeval
❑ Shimonishi, Hideyuki, M. Y. Sanadidi, and Tutomu Murase. “Assessing
Interactions among Legacy and High-Speed TCPs.” PFLDnet 2007 (2007).
- designed for evaluating High-speed TCP extensions using ns-2
- Source: http://nrlweb.cs.ucla.edu/tcpsuite/index.html
15th June 2016, Wednesday University of Washington, Seattle, USA4
Existing implementations
❑ Li, Yee-Ting, Douglas Leith, and Robert N. Shorten. “Experimental
evaluation of TCP protocols for high-speed networks.” Networking, IEEE/
ACM Transactions on 15.5 (2007): 1109-1122.
- designed for evaluating High-speed TCP extensions using ns-2
- Source: http://www.hamilton.ie/net/eval/hi2005.htm
❑ Hayes, D., Ros, D., Andrew, L. and S. FLoyd, “Common TCP Evaluation
Suite” draft-irtf-iccrg-tcpeval-01, IETF Internet Draft (expired) (2015).
- The latest draft on TCP Evaluation Suite
- Source: https://bitbucket.org/hayesd/tcp-evaluation-suite-public
15th June 2016, Wednesday University of Washington, Seattle, USA5
Design and implementation of ns-3 tcp-eval
❑ Implemented as a separate model called tcp-eval in ns-3 (~5500 lines)
❑ Topologies:
- Dumbbell (single bottleneck topology)
- Parking lot (multiple bottlenecks topology)
❑ Traffic types:
- Long lived FTP
- Streaming video
- Interactive voice
❑ Performance metrics:
- Aggregate link utilization
- Mean queue length, and Packet drop rate
15th June 2016, Wednesday University of Washington, Seattle, USA6
Design and implementation of ns-3 tcp-eval
15th June 2016, Wednesday University of Washington, Seattle, USA
Figure: Class diagram of tcp-eval in ns-3
7
User interaction with ns-3 tcp-eval
15th June 2016, Wednesday University of Washington, Seattle, USA
Figure: User interaction diagram of tcp-eval for dumbbell scenario
User
drive-dumbbell.cc
Results
point-to-point-dumbbell.ccdumbbell-topology.cc
eval-stats.cc
create-traffic.cc
traffic-parameters.cc
configure-topology.cc
1
2
3
4
5
6
7
8Core modules
Configuration module
Supporting module
Legends
8
User interaction with ns-3 tcp-eval
15th June 2016, Wednesday University of Washington, Seattle, USA
Figure: User interaction diagram of tcp-eval for parking-lot scenario
User
drive-parking-lot.cc
Results
point-to-point-parking-lot.ccparking-lot-topology.cc
eval-stats.cc
create-traffic.cc
traffic-parameters.cc
configure-topology.cc
1
2
3
4
5
6
7
8
Core modules
Configuration module
Supporting module
Legends
9
Comparing TCP extensions in ns-3
❑ Five TCP extensions: Tahoe, Reno, NewReno, Westwood, Westwood+
❑ Three scenarios:
- Varying bottleneck bandwidth
- Varying RTT
- Varying the number of FTP flows
❑ Three Performance metrics:
- Link utilization
- Mean queue length
- Packet drop rate
❑ Output:
- PDF containing graphs (LaTex must be installed!)
15th June 2016, Wednesday University of Washington, Seattle, USA10
Results and discussions: varying bottleneck bandwidth
15th June 2016, Wednesday University of Washington, Seattle, USA
0
20
40
60
80
100
1 10 100
Lin
k U
tiliz
atio
n (
%)
Bandwidth (Mbps) Log Scale
Link Utilization with Bandwidth Changes
TahoeReno
NewrenoWestwood
Westwood+ 0
20
40
60
80
100
1 10 100
Lin
k U
tiliz
atio
n (
%)
Bandwidth (Mbps) Log Scale
Link Utilization with Bandwidth Changes
TahoeReno
NewrenoWestwood
Westwood+
Dumbbell topology Parking lot topology
11
Results and discussions: varying bottleneck bandwidth
15th June 2016, Wednesday University of Washington, Seattle, USA
Dumbbell topology Parking lot topology
0
20
40
60
80
100
1 10 100
Mean Q
ueue
Length
(%
)
Bandwidth (Mbps) Log Scale
Percent of Mean Queue Length with Bandwidth Changes
TahoeReno
NewrenoWestwood
Westwood+
0
20
40
60
80
100
1 10 100
Mean Q
ueue
Length
(%
)
Bandwidth (Mbps) Log Scale
Percent of Mean Queue Length with Bandwidth Changes
TahoeReno
NewrenoWestwood
Westwood+
12
Results and discussions: varying bottleneck bandwidth
15th June 2016, Wednesday University of Washington, Seattle, USA
Dumbbell topology Parking lot topology
0
2
4
6
8
1 10 100
Pack
et D
rop R
ate
(%
)
Bandwidth (Mbps) Log Scale
Packet Drop Rate with Bandwidth Changes
TahoeReno
NewrenoWestwood
Westwood+
0
2
4
6
8
1 10 100
Pack
et D
rop R
ate
(%
)
Bandwidth (Mbps) Log Scale
Packet Drop Rate with Bandwidth Changes
TahoeReno
NewrenoWestwood
Westwood+
13
Results and discussions: varying RTT
15th June 2016, Wednesday University of Washington, Seattle, USA
Dumbbell topology Parking lot topology
0
20
40
60
80
100
0.01 0.1 1
Lin
k U
tiliz
atio
n (
%)
RTT (s) Log Scale
Link Utilization with RTT Changes
TahoeReno
NewrenoWestwood
Westwood+ 0
20
40
60
80
100
0.01 0.1 1
Lin
k U
tiliz
atio
n (
%)
RTT (s) Log Scale
Link Utilization with RTT Changes
TahoeReno
NewrenoWestwood
Westwood+
14
Results and discussions: varying RTT
15th June 2016, Wednesday University of Washington, Seattle, USA
Dumbbell topology Parking lot topology
0
10
20
30
40
50
60
70
80
90
100
0.01 0.1 1
Mean Q
ueue
Length
(%
)
RTT (s) Log Scale
Percent of Mean Queue Length with RTT Changes
TahoeReno
NewrenoWestwood
Westwood+
0
10
20
30
40
50
60
70
80
90
100
0.01 0.1 1
Mean Q
ueue
Length
(%
)
RTT (s) Log Scale
Percent of Mean Queue Length with RTT Changes
TahoeReno
NewrenoWestwood
Westwood+
15
Results and discussions: varying RTT
15th June 2016, Wednesday University of Washington, Seattle, USA
Dumbbell topology Parking lot topology
0
2
4
6
8
0.01 0.1 1
Pack
et D
rop R
ate
(%
)
RTT (s) Log Scale
Packet Drop Rate with RTT Changes
TahoeReno
NewrenoWestwood
Westwood+
0
2
4
6
8
0.01 0.1 1
Pack
et D
rop R
ate
(%
)
RTT (s) Log Scale
Packet Drop Rate with RTT Changes
TahoeReno
NewrenoWestwood
Westwood+
16
Results and discussions: varying number of FTP flows
15th June 2016, Wednesday University of Washington, Seattle, USA
Dumbbell topology Parking lot topology
0
20
40
60
80
100
1 10 100
Lin
k U
tiliz
atio
n (
%)
FTP Log Scale
Link Utilization with FTP Changes
TahoeReno
NewrenoWestwood
Westwood+ 0
20
40
60
80
100
1 10 100
Lin
k U
tiliz
atio
n (
%)
FTP Log Scale
Link Utilization with FTP Changes
TahoeReno
NewrenoWestwood
Westwood+
17
Results and discussions: varying number of FTP flows
15th June 2016, Wednesday University of Washington, Seattle, USA
Dumbbell topology Parking lot topology
0
20
40
60
80
100
1 10 100
Mean Q
ueue
Length
(%
)
FTP Log Scale
Percent of Mean Queue Length with FTP Changes
TahoeReno
NewrenoWestwood
Westwood+ 0
20
40
60
80
100
1 10 100
Mean Q
ueue
Length
(%
)
FTP Log Scale
Percent of Mean Queue Length with FTP Changes
TahoeReno
NewrenoWestwood
Westwood+
18
Results and discussions: varying number of FTP flows
15th June 2016, Wednesday University of Washington, Seattle, USA
Dumbbell topology Parking lot topology
0
2
4
6
8
1 10 100
Pack
et D
rop R
ate
(%
)
FTP Log Scale
Packet Drop Rate with FTP Changes
TahoeReno
NewrenoWestwood
Westwood+
0
2
4
6
8
1 10 100
Pack
et D
rop R
ate
(%
)
FTP Log Scale
Packet Drop Rate with FTP Changes
TahoeReno
NewRenoWestwood
Westwood+
19
Validation issues
❑ Original tcp-eval is implemented in older version of ns-2 (ns-2.31!)
❑ ns-2.31 did not have many new TCPs
❑ Hence, tcp-eval contained custom implementations of new TCPs
❑ Latest tcp-eval implementation in ns-2 is on ns-2.35
❑ But there are several bugs identified, and its development has stopped
❑ Started aligning our implementation with that of tcp-eval for ns-2.35
15th June 2016, Wednesday University of Washington, Seattle, USA20
Conclusions and the next goals
❑ A ns-3 model for tcp-eval has been implemented, but not validated.
❑ Automates the cycle from setting parameters to collecting results
❑ Steps to reproduce the results have been provided.
Next goals:
❑ Align the model to latest version of tcp-eval (2016 summer project!)
❑ Evaluate the model by comparing its results to those obtained from ns-2
❑ Include support for more topologies (wireless) and AQM algorithms
❑ Provide per-flow analysis to the user.
15th June 2016, Wednesday University of Washington, Seattle, USA21
Acknowledgement
❑ All the reviewers, for the encouraging reviews!
❑ Tom Henderson, for guiding us through the validation procedure.
❑ All developers of TCP extensions in ns-3.
❑ The entire ns-3 community.
❑ Our research group at NITK Surathkal.
15th June 2016, Wednesday University of Washington, Seattle, USA22
Thank you.
