indianauniversityindianauniversity transpac qbss “scavenger service” chris robb indiana...

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

TransPAC QBSS“Scavenger Service”

Chris Robb

Indiana University

TransPAC Network Engineer

[email protected]

4 October 2001TransPAC Juniper QBSS Implementation

Chris Robb – Indiana University Global [email protected]

I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Topics

QBSS on TransPAC Scavenger Availability Juniper Implementation Juniper Testing Learnings Future Directions Questions and Comments



I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Scavenger on TransPAC

The need to implement QBSS on the TransPAC circuit came about after discussions with a group of researchers from Indiana University and the University of Tokyo.

A GRAPE-6 machine at Tokyo will be generating terabytes of data Indiana University is supplying an HPSS archive to store the data

for analysis by other US researches. This data will need to be sent across the TransPAC OC-3 in a

fashion that wouldn’t cause any significant network impact Researchers agreed to tag their IP datagrams as QBSS so they

wouldn’t interrupt the other Tokyo-Chicago traffic. This project is slated to begin transfers very soon.



I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Where is Scavenger Available?

TransPAC circuit from Tokyo to Chicago– We will focus on this link because it focuses on a

Juniper implementation



I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Juniper Implementation

Both sides of the TransPAC link were serviced by Juniper routers, so a Juniper implementation of Scavenger Service needed to be created and tested.

Juniper proposed an implementation and the Global NOC volunteered to test its functionality.



I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Juniper Test Setup

Juniper donated an M5 for general Global NOC testing in June of 2001

The test setup was as follows:

Both workstations were Linux-based. JunOS 4.3 was loaded on the M5



I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Juniper Test Setup (cont.)

Juniper doesn’t yet have DSCP support, so TOS bits were used instead. The I2 QBSS implementation accounts for this and Juniper has said that DSCP support is forthcoming.

A precedence map was created to place the tagged packets into their appropriate queues. This remained fairly constant throughout the testing: precedence-map QBSS-test {

bits 000 output-queue 0; <- Best Effort Traffic

bits 001 output-queue 1; <- QBSS tagged traffic

bits 101 output-queue 0;


bits 011 output-queue 2; <- Premium traffic


bits 110 output-queue 3; <- Network Management Traffic. Juniper advises

bits 111 output-queue 3; <- not to change the queue assignment for these

}



I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y


After the traffic is placed in the queue, a WRR profile is created for the individual interfaces.

The following WRR profile was created to service the outgoing ATM interface:

weighted-round-robin { output-queue 0 weight 98; <- best-effort traffic output-queue 1 weight 1; <- QBSS gets a minimum of 1% of the circuit output-queue 2 weight 0; <- Premium service receives no servicing output-queue 3 weight 1; <- Network Management traffic must have some servicing }

This configuration would change, depending on what we wanted to test. In the above configuration, we weren’t testing premium service



I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y


A patched version of iPerf that allowed for TOS tagging was used to generate multiple streams from the GigE NIC towards the ATM NIC.

Most of the tests involved running 200Mbit UDP streams of one type, and then introducing another stream of the other type

The results were logged by iPerf at one second intervals, providing a picture of overall traffic behavior during the particular test.



I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Test results

The overall results were positive! The following graph shows the results of one particular test:

At +0s, a 200Mb QBSS stream was run. At +10s, a best-effort stream was introduced, pushing down the QBSS stream to about 1 percent of the available bandwidth. At + 40s, the BE stream was removed and the QBSS stream reclaimed the pipe.



I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Test Results (cont.)

Here’s another graph showing a 200Mb QBSS stream before and after the introduction of a 10Mbit best-effort stream:

The full results of these tests can be viewed at:http://www.transpac.org/qbss-html



I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Learnings

Juniper will only use the WRR queuing profile when a circuit is saturated.

Because you cannot configure WRR queuing on a per-VC basis, this has wide implications for small VCs.

– For example, a policed 10Mbit VC will never fill an entire OC-3 ATM PIC.

– QBSS traffic on that VC will not be serviced any differently than best-effort traffic unless the circuit is filled up with traffic from other configured VCs

One way to get around this is to generate traffic on the other VCs to artificially fill up the circuit, although this is highly undesirable.



I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Learnings (cont.)

Fortunately, Juniper does have a way around this!– There is a “hidden” command in the JunOS that will force all

traffic on an interface, regardless of the link saturation, to be subject to the WRR profile:

set chassis fpc <x> pic <y> transmit-buffers <n>

It is also important to set queue lengths for the individual VCs on the circuit to make sure that a small VC doesn’t eat up all the buffer space when delaying QBSS traffic.



I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Learnings (cont.)

Expedited traffic must be placed in queue 0– When swapping the best effort and QBSS queues around (i.e.

QBSS in queue 0 and best-effort in queue 1), QBSS traffic did not yield to best-effort traffic

– Instead, they evenly split the circuit, with both flows getting half of the bandwidth

– We spoke at length with Juniper about this problem. Unfortunately, we had to return the M5 before this was resolved.

– It isn’t crippling, but something to keep in mind



I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

TransPAC Implementation

Thanks to the APAN NOC, TransPAC is now enabled for QBSS transmissions, using the WRR profile shown earlier.

Initial tests have proven positive, but more testing is needed before the GRAPE transfers are initiated.



I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Future Directions

Results must be shared with QoS working group QBSS mailing list

More complete testing of the “queue 0” problem is needed

We also need to test TCP performance IU has recently reacquired the hardware to

complete such tests. Results will be shared.



I

N

D

I

A

N

A

U

N

I

V

E

R

S

I

T

Y

Questions? Comments?

indianauniversityindianauniversity transpac qbss “scavenger service” chris robb indiana...

Documents