15th June 2016, Wednesday University of Washington, Seattle, USA

Mohit P. Tahiliani, Dharmendra Kumar Mishra, Pranav Vankar

Wireless Information Networking Group,

National Institute of Technology Karnataka, Surathkal, India

tahiliani@nitk.ac.in

8th Annual Workshop on ns-3 (WNS3 2016)

TCP Evaluation Suite for ns-3

Outline of the presentation

❑ Motivation

❑ Existing implementations

❑ Design and implementation of tcp-eval in ns-3

❑ User interaction with ns-3 tcp-eval

❑ Comparing TCP extensions in ns-3

❑ Results and discussions

❑ Validation issues

❑ Conclusions and the next goals

15th June 2016, Wednesday University of Washington, Seattle, USA

Motivation

❑ Congestion control algorithms continue to evolve . . .

❑ . . . and so do TCP extensions!

❑ Problem: not feasible to evaluate every TCP extension exhaustively

❑ Potential solution:

- derive some initial results and study the behaviour

- consider the promising ones for thorough evaluation

What is TCP Evaluation Suite?

- a set of well-defined, standard test cases to compare TCP extensions

- initially proposed by Transport Modeling Research Group (TMRG)

- modified by Internet Congestion Control Research Group (ICCRG)

- widely used today for evaluating new TCP extensions

15th June 2016, Wednesday University of Washington, Seattle, USA3

Existing implementations

❑ Wang, G., Y. Xia, and D. Harrison. “An NS2 TCP evaluation tool.”

draftirtf-tmrg-ns2-tcp-tool, IETF Internet Draft (expired) (2007).

- Two versions of code.

- Version 2 source: https://sourceforge.net/projects/tcpeval

❑ Shimonishi, Hideyuki, M. Y. Sanadidi, and Tutomu Murase. “Assessing

Interactions among Legacy and High-Speed TCPs.” PFLDnet 2007 (2007).

- designed for evaluating High-speed TCP extensions using ns-2

- Source: http://nrlweb.cs.ucla.edu/tcpsuite/index.html

15th June 2016, Wednesday University of Washington, Seattle, USA4

Existing implementations

❑ Li, Yee-Ting, Douglas Leith, and Robert N. Shorten. “Experimental

evaluation of TCP protocols for high-speed networks.” Networking, IEEE/

ACM Transactions on 15.5 (2007): 1109-1122.

- designed for evaluating High-speed TCP extensions using ns-2

- Source: http://www.hamilton.ie/net/eval/hi2005.htm

❑ Hayes, D., Ros, D., Andrew, L. and S. FLoyd, “Common TCP Evaluation

Suite” draft-irtf-iccrg-tcpeval-01, IETF Internet Draft (expired) (2015).

- The latest draft on TCP Evaluation Suite

- Source: https://bitbucket.org/hayesd/tcp-evaluation-suite-public

15th June 2016, Wednesday University of Washington, Seattle, USA5

Design and implementation of ns-3 tcp-eval

❑ Implemented as a separate model called tcp-eval in ns-3 (~5500 lines)

❑ Topologies:

- Dumbbell (single bottleneck topology)

- Parking lot (multiple bottlenecks topology)

❑ Traffic types:

- Long lived FTP

- Streaming video

- Interactive voice

❑ Performance metrics:

- Aggregate link utilization

- Mean queue length, and Packet drop rate

15th June 2016, Wednesday University of Washington, Seattle, USA6

Design and implementation of ns-3 tcp-eval

15th June 2016, Wednesday University of Washington, Seattle, USA

Figure: Class diagram of tcp-eval in ns-3

7

User interaction with ns-3 tcp-eval

15th June 2016, Wednesday University of Washington, Seattle, USA

Figure: User interaction diagram of tcp-eval for dumbbell scenario

User

drive-dumbbell.cc

Results

point-to-point-dumbbell.ccdumbbell-topology.cc

eval-stats.cc

create-traffic.cc

traffic-parameters.cc

configure-topology.cc

1

2

3

4

5

6

7

8Core modules

Configuration module

Supporting module

Legends

8

User interaction with ns-3 tcp-eval

15th June 2016, Wednesday University of Washington, Seattle, USA

Figure: User interaction diagram of tcp-eval for parking-lot scenario

User

drive-parking-lot.cc

Results

point-to-point-parking-lot.ccparking-lot-topology.cc

eval-stats.cc

create-traffic.cc

traffic-parameters.cc

configure-topology.cc

1

2

3

4

5

6

7

8

Core modules

Configuration module

Supporting module

Legends

9

Comparing TCP extensions in ns-3

❑ Five TCP extensions: Tahoe, Reno, NewReno, Westwood, Westwood+

❑ Three scenarios:

- Varying bottleneck bandwidth

- Varying RTT

- Varying the number of FTP flows

❑ Three Performance metrics:

- Link utilization

- Mean queue length

- Packet drop rate

❑ Output:

- PDF containing graphs (LaTex must be installed!)

15th June 2016, Wednesday University of Washington, Seattle, USA10

Results and discussions: varying bottleneck bandwidth

15th June 2016, Wednesday University of Washington, Seattle, USA

0

20

40

60

80

100

1 10 100

Lin

k U

tiliz

atio

n (

%)

Bandwidth (Mbps) Log Scale

Link Utilization with Bandwidth Changes

TahoeReno

NewrenoWestwood

Westwood+ 0

20

40

60

80

100

1 10 100

Lin

k U

tiliz

atio

n (

%)

Bandwidth (Mbps) Log Scale

Link Utilization with Bandwidth Changes

TahoeReno

NewrenoWestwood

Westwood+

Dumbbell topology Parking lot topology

11

Results and discussions: varying bottleneck bandwidth

15th June 2016, Wednesday University of Washington, Seattle, USA

Dumbbell topology Parking lot topology

0

20

40

60

80

100

1 10 100

Mean Q

ueue

Length

(%

)

Bandwidth (Mbps) Log Scale

Percent of Mean Queue Length with Bandwidth Changes

TahoeReno

NewrenoWestwood

Westwood+

0

20

40

60

80

100

1 10 100

Mean Q

ueue

Length

(%

)

Bandwidth (Mbps) Log Scale

Percent of Mean Queue Length with Bandwidth Changes

TahoeReno

NewrenoWestwood

Westwood+

12

Results and discussions: varying bottleneck bandwidth

15th June 2016, Wednesday University of Washington, Seattle, USA

Dumbbell topology Parking lot topology

0

2

4

6

8

1 10 100

Pack

et D

rop R

ate

(%

)

Bandwidth (Mbps) Log Scale

Packet Drop Rate with Bandwidth Changes

TahoeReno

NewrenoWestwood

Westwood+

0

2

4

6

8

1 10 100

Pack

et D

rop R

ate

(%

)

Bandwidth (Mbps) Log Scale

Packet Drop Rate with Bandwidth Changes

TahoeReno

NewrenoWestwood

Westwood+

13

Results and discussions: varying RTT

15th June 2016, Wednesday University of Washington, Seattle, USA

Dumbbell topology Parking lot topology

0

20

40

60

80

100

0.01 0.1 1

Lin

k U

tiliz

atio

n (

%)

RTT (s) Log Scale

Link Utilization with RTT Changes

TahoeReno

NewrenoWestwood

Westwood+ 0

20

40

60

80

100

0.01 0.1 1

Lin

k U

tiliz

atio

n (

%)

RTT (s) Log Scale

Link Utilization with RTT Changes

TahoeReno

NewrenoWestwood

Westwood+

14

Results and discussions: varying RTT

15th June 2016, Wednesday University of Washington, Seattle, USA

Dumbbell topology Parking lot topology

0

10

20

30

40

50

60

70

80

90

100

0.01 0.1 1

Mean Q

ueue

Length

(%

)

RTT (s) Log Scale

Percent of Mean Queue Length with RTT Changes

TahoeReno

NewrenoWestwood

Westwood+

0

10

20

30

40

50

60

70

80

90

100

0.01 0.1 1

Mean Q

ueue

Length

(%

)

RTT (s) Log Scale

Percent of Mean Queue Length with RTT Changes

TahoeReno

NewrenoWestwood

Westwood+

15

Results and discussions: varying RTT

15th June 2016, Wednesday University of Washington, Seattle, USA

Dumbbell topology Parking lot topology

0

2

4

6

8

0.01 0.1 1

Pack

et D

rop R

ate

(%

)

RTT (s) Log Scale

Packet Drop Rate with RTT Changes

TahoeReno

NewrenoWestwood

Westwood+

0

2

4

6

8

0.01 0.1 1

Pack

et D

rop R

ate

(%

)

RTT (s) Log Scale

Packet Drop Rate with RTT Changes

TahoeReno

NewrenoWestwood

Westwood+

16

Results and discussions: varying number of FTP flows

15th June 2016, Wednesday University of Washington, Seattle, USA

Dumbbell topology Parking lot topology

0

20

40

60

80

100

1 10 100

Lin

k U

tiliz

atio

n (

%)

FTP Log Scale

Link Utilization with FTP Changes

TahoeReno

NewrenoWestwood

Westwood+ 0

20

40

60

80

100

1 10 100

Lin

k U

tiliz

atio

n (

%)

FTP Log Scale

Link Utilization with FTP Changes

TahoeReno

NewrenoWestwood

Westwood+

17

Results and discussions: varying number of FTP flows

15th June 2016, Wednesday University of Washington, Seattle, USA

Dumbbell topology Parking lot topology

0

20

40

60

80

100

1 10 100

Mean Q

ueue

Length

(%

)

FTP Log Scale

Percent of Mean Queue Length with FTP Changes

TahoeReno

NewrenoWestwood

Westwood+ 0

20

40

60

80

100

1 10 100

Mean Q

ueue

Length

(%

)

FTP Log Scale

Percent of Mean Queue Length with FTP Changes

TahoeReno

NewrenoWestwood

Westwood+

18

Results and discussions: varying number of FTP flows

15th June 2016, Wednesday University of Washington, Seattle, USA

Dumbbell topology Parking lot topology

0

2

4

6

8

1 10 100

Pack

et D

rop R

ate

(%

)

FTP Log Scale

Packet Drop Rate with FTP Changes

TahoeReno

NewrenoWestwood

Westwood+

0

2

4

6

8

1 10 100

Pack

et D

rop R

ate

(%

)

FTP Log Scale

Packet Drop Rate with FTP Changes

TahoeReno

NewRenoWestwood

Westwood+

19

Validation issues

❑ Original tcp-eval is implemented in older version of ns-2 (ns-2.31!)

❑ ns-2.31 did not have many new TCPs

❑ Hence, tcp-eval contained custom implementations of new TCPs

❑ Latest tcp-eval implementation in ns-2 is on ns-2.35

❑ But there are several bugs identified, and its development has stopped

❑ Started aligning our implementation with that of tcp-eval for ns-2.35

15th June 2016, Wednesday University of Washington, Seattle, USA20

Conclusions and the next goals

❑ A ns-3 model for tcp-eval has been implemented, but not validated.

❑ Automates the cycle from setting parameters to collecting results

❑ Steps to reproduce the results have been provided.

Next goals:

❑ Align the model to latest version of tcp-eval (2016 summer project!)

❑ Evaluate the model by comparing its results to those obtained from ns-2

❑ Include support for more topologies (wireless) and AQM algorithms

❑ Provide per-flow analysis to the user.

15th June 2016, Wednesday University of Washington, Seattle, USA21

Acknowledgement

❑ All the reviewers, for the encouraging reviews!

❑ Tom Henderson, for guiding us through the validation procedure.

❑ All developers of TCP extensions in ns-3.

❑ The entire ns-3 community.

❑ Our research group at NITK Surathkal.

15th June 2016, Wednesday University of Washington, Seattle, USA22

Thank you.