Download - Scheduling Techniques for Hybrid Circuit/Packet …./mmukerje/Solstice-Conext15-Final.pdf · Scheduling Techniques for Hybrid Circuit/Packet Networks ... Circuit Switching Packet

Scheduling Techniques for Hybrid Circuit/Packet

NetworksHe Liu, Matthew K. Mukerjee, Conglong Li, Nicolas Feltman,

George Papen, Stefan Savage, Srinivasan Seshan, Geoffrey M. Voelker, David G. Andersen, Michael Kaminsky, George Porter, and Alex C. Snoeren

Hybrid circuit/packet networks: Can we get best of both?

Circuit Switching Packet Switching

High link rate

Low link rate

Limits on num flows+ Explicit flow setup

Any-to-any communication

This talk: commun. restricted to pairs

+ non-trivial circuit reconfig time

Hybrid networks in datacenters*

*Helios [SIGCOMM ’10], c-Through [SIGCOMM ’10], Flyways [SIGCOMM ’11], …

RACK RACK RACK RACK RACK RACK RACK RACK

Circuit Packet



A B CABC

Datacenter workloadsare skewed

Senders

Receivers

Senders

ReceiversHybrid networks in datacenters*


A B CABC

Prior work showsmajor benefits

(“Keep circuits busy!”)



Packet switch as “escape hatch”

for difficult demands

Need an explicit sequence of circuit

configurations


How do proposed hybrid networks schedule demand?They don’t! They use an oracle


Does prior scheduling work help?

iSLIP BvN

Works per-packet (can’t identify “difficult” to schedule demands)

Works on all demand simultaneously

(can identify “difficult” demands)

Both assume circuit reconfig is free (use many configurations)

(Fast heuristic per-packet) Optimal

Does prior scheduling work help?BvN

Both assume circuit reconfig is free (use many configurations)

Time

iSLIP

Contribution: Solstice

• Characterizing

• Optimal formulation and bounds

• A fast, heuristic-based hybrid scheduler “ ”

• of Solstice: performs 2-3x better than previous solutions and within 14% of optimal up to a 128-port switch

the hybrid scheduling problem

Solstice

Evaluation

Outline

• Model

•

•

•

the hybrid scheduling problem

Solstice

Evaluation

Goal of hybrid scheduling

?

A B CABC

Schedule all demand in minimum duration

Goal of hybrid scheduling

Schedule all demand in minimum duration

duration = total circuit transmission time+

number of configurations * circuit reconfig cost

Configuration selection and duration

What goes topacket switch

The hybrid scheduling problem

?

A B CABC

Understanding circuitsKey properties

of circuit switching

Configuration

Duration

A B CABC

Understanding circuitsKey properties

of circuit switching

Configuration

Duration

Src Dest

A

B

C

A

B

C

A B CABC

A

B

C

A

B

C

Src Dest

Configuration

Duration


?Circuit Switch Packet Switch

A

B

C

A

B

C

A B CABC


?Circuit Switch Packet Switch

A

B

C

A

B

C

A B CABC

A B CABC

How to compute the output?

Outline

• Model

• The hybrid scheduling problem

• Solstice

• Evaluation

Solstice: intuition

• What can we take advantage of?

• Skew in workload (i.e., large + small demands)

• Circuit switch is efficient for large demands

• Packet switch is efficient for small demands

Solstice: key ideaSchedule largedemands first

on circuit

A B CABC

D

D

Solstice: key ideaSchedule largedemands first

on circuit

Circuit SwitchLoop

A B CABC

D

D

A

B

C

A

B

C

D D

Solstice: key idea

Circuit Switch

A B CABC

D

D

Schedule largedemands first

on circuitLoop

A

B

C

A

B

C

D D

A

B

C

A

B

C

D D

Solstice: key ideaTerminate when

remainder can be served by

packet switch

Circuit Switch

A B CABC

D

D

A

B

C

A

B

C

D D

A

B

C

A

B

C

D D

Packet SwitchA B C

ABC

D

D

Solstice: key idea

Large demand alwayson circuit switch

(efficient)

Small demand alwayson packet switch

(efficient)Circuit SwitchA

B

C

A

B

C

D D

A

B

C

A

B

C

D D

Packet SwitchA B C

ABC

D

D

Solstice: example

A B CABC

D E

DE

Solstice: exampleTotal

Duration

0Reconfig time = 2.5

. 5 40 . 5 5 . . 40 5 . . . 10 4045 5 . . . . 40 10 . .

Solstice: exampleServableon Packet

0Total

Duration

0Reconfig time = 2.5 Packet BW = 1/10

. 5 40 . 5 5 . . 40 5 . . . 10 4045 5 . . . . 40 10 . .


0Threshold

32Total

Duration


. 5 40 . 5 5 . . 40 5 . . . 10 4045 5 . . . . 40 10 . .

. 5 40 . 5 5 . . 40 5 . . . 10 4045 5 . . . . 40 10 . .

A

B

C

A

B

C

D

E

D

E


0Threshold

32Total

Duration


40


0Threshold

32Total

Duration


40

A

B

C

A

B

C

D

E

D

E . 5 . . 5 5 . . . 5 . . . 10 . 5 5 . . . . . 10 . .


0Threshold

32Total

Duration

42.5Reconfig time = 2.5 Packet BW = 1/10

40

A

B

C

A

B

C

D

E

D

E . 5 . . 5 5 . . . 5 . . . 10 . 5 5 . . . . . 10 . .


4.25Threshold

32Total

Duration


40

A

B

C

A

B

C

D

E

D

E . 5 . . 5 5 . . . 5 . . . 10 . 5 5 . . . . . 10 . .


4.25Threshold

16Total

Duration


40

A

B

C

A

B

C

D

E

D

E . 5 . . 5 5 . . . 5 . . . 10 . 5 5 . . . . . 10 . .

Threshold too large


4.25Threshold

8Total

Duration


40

A

B

C

A

B

C

D

E

D

E . 5 . . 5 5 . . . 5 . . . 10 . 5 5 . . . . . 10 . .

Solstice: example

A

B

C

A

B

C

D

E

D

E

Servableon Packet

4.25Threshold

8Total

Duration


40

A

B

C

A

B

C

D

E

D

E . 5 . . 5 5 . . . 5 . . . 10 . 5 5 . . . . . 10 . .

Incompleteconfiguration


4.25Threshold

4Total

Duration


40

A

B

C

A

B

C

D

E

D

E . 5 . . 5 5 . . . 5 . . . 10 . 5 5 . . . . . 10 . .

Solstice: example

A

B

C

A

B

C

D

E

D

E

Servableon Packet

4.25Threshold

4Total

Duration


40

A

B

C

A

B

C

D

E

D

E

A

B

C

A

B

C

D

E

D

E

. 5 . . 5 5 . . . 5 . . . 10 . 5 5 . . . . . 10 . .

MultipleConfigs

. 5 . . 5 5 . . . 5 . . . 10 . 5 5 . . . . . 10 . .

A

B

C

A

B

C

D

E

D

E


4.25Threshold

4Total

Duration


40

A

B

C

A

B

C

D

E

D

E

5


4.25Threshold

4Total

Duration


40

A

B

C

A

B

C

D

E

D

E

5

A

B

C

A

B

C

D

E

D

E . 5 . . . . . . . 5 . . . 5 . 5 . . . . . . 5 . .


5Threshold

4Total

Duration


40

A

B

C

A

B

C

D

E

D

E

5

A

B

C

A

B

C

D

E

D

E

. 5 . . . . . . . 5 . . . 5 . 5 . . . . . . 5 . .

. 5 . . . . . . . 5 . . . 5 . 5 . . . . . . 5 . .

Servableon Packet

5

Solstice: exampleThreshold

4

. . . . . . . . . . . . . . . . . . . . . . . . .

. 5 . . . . . . . 5 . . . 5 . 5 . . . . . . 5 . .

Total Duration


40

A

B

C

A

B

C

D

E

D

E

5

A

B

C

A

B

C

D

E

D

E

Servableon Packet

5


4

. . . . . . . . . . . . . . . . . . . . . . . . .

. 5 . . . . . . . 5 . . . 5 . 5 . . . . . . 5 . .

Total Duration


40

A

B

C

A

B

C

D

E

D

E

5

A

B

C

A

B

C

D

E

D

E

ScheduledAll Demand

Servableon Packet

5


4

. . . . . . . . . . . . . . . . . . . . . . . . .

Total Duration


40

A

B

C

A

B

C

D

E

D

E

5

A

B

C

A

B

C

D

E

D

E

ScheduledAll Demand

2 Configs

. 5 . . . . . . . 5 . . . 5 . 5 . . . . . . 5 . .Remainderon Packet

Outline

• Model

• The hybrid scheduling problem

• Solstice

• Evaluation

Inefficiencies

• Two possible ways to be inefficient

• Circuit reconfig “downtime”

• Link is active but sender has no data to send

• Focus on first one in this talk; second in paper

Metrics

• Total schedule duration

• total circuit transmission time + number of configurations * reconfig time

• Number of circuit configurations used in schedule

8 12 16 32 64 128Number of ports

0

2

4

6

8

10

12

14

16

Tim

e(m

s)Schedule duration

Better

Worse


0

2

4

6

8

10

12

14

16

Tim

e(m

s)Schedule duration

Better

Worse

Lower Bound


0

2

4

6

8

10

12

14

16

Tim

e(m

s)Schedule duration

Upper Bound

Better

Worse

Lower Bound


0

2

4

6

8

10

12

14

16

Tim

e(m

s)

Solstice

Schedule duration

Solstice at most 14% from optimal for these counts

Better

Worse


0

2

4

6

8

10

12

14

16

Tim

e(m

s)

Solstice BvN

Schedule duration

Better

Worse


0

2

4

6

8

10

12

14

16

Tim

e(m

s)

Solstice BvN iSLIP

Schedule durationSolstice

2-3x better

iSLIP poor overall. Why?

BvN gets worse here. Why?

Better

Worse


0

50

100

150

200

Num

confi

gs

Solstice BvN iSLIP

Number of configurations

iSLIP uses many configurations

throughout

BvN almost 3x number of

configurations

Better

Worse

Other results in paper

• Effects of skew (up to 10% worse)

• Effects of sparsity (up to 26% worse)

• Runtime complexity analysis

• Augmenting Solstice with offline computation

Conclusion

• Characterizing the hybrid scheduling problem

• Optimal formulation and bounds

• A fast, heuristic-based hybrid scheduler “Solstice”

• Evaluation of Solstice: performs 2-3x better than previous solutions and within 14% of optimal up to a 128-port switch

Scheduling Techniques for Hybrid Circuit/Packet

NetworksHe Liu, Matthew K. Mukerjee, Conglong Li, Nicolas Feltman,

George Papen, Stefan Savage, Srinivasan Seshan, Geoffrey M. Voelker, David G. Andersen, Michael Kaminsky, George Porter, and Alex C. Snoeren

Backup Slides…

Solving Optimally• Use an ILP to solve the problem

• Provides intuition and a lower bound (i.e., this is the best you can do)

• has to work through n! possible configurations

• Slow (e.g., 12-port takes minutes, 16-port takes hours)

Stuffing . 13 10 70 4 . . 14 12 6965 . . 12 1415 33 2 . 1112 7 3 1 .

. 14 10 70 4 . . 17 12 6971 . . 13 1415 70 2 . 1112 14 69 3 .

= Variable row/col sum (but max is 98)

Fixed row/col sum (98) =

Pick large entries and increase until maxed

Runtime complexity• Dominated by circuit configuration selection

• (equivalent to finding a perfect matching in a bipartite graph)

• O(n * non-zero entries in rows) + O(n * log2 n)

• If we assume number of non-zero entries is constant (in line with prior work on dc workloads), total is O(n2 * log2 n)

Simulation Setup• 20 microsecond reconfiguration time

• accumulation period of 3 ms, schedule all traffic

• 100Gbps circuit links, 10Gbps packet links

• sparse matrices (4 large and 12 small demands per port on 64 port switch)

• skewed matrices (small demands total 30% of demand)

• 100 random matrices for testing


0

20

40

60

80

100

Util

izat

ion

(%)

iSLIP BvN Solstice

Circuit utilizationNumber of configshas great impact

on circuit utilization

Worse

Better

8 16 24 32 40 48 56 64Number of demands per port

2.02.53.03.54.04.5

Tim

e(m

s)

2.8 3.

1 3.3 3.

5 3.6

3.7

3.7 3.

9Solstice

Sparsity

Default

Worse

Better

only 26% worse

. 3 . . . . . 4 . . . . . 4 . . . . . 142 . . . .

32 13 10 70 424 29 14 12 6965 1 10 12 1415 33 2 42 1112 7 3 1 3

Skew

0 5 10 15 20 25 30 35 40 45 50Bandwidth requested by small flows (%)

2.02.53.03.54.04.5

Tim

e(m

s)

2.7 2.

9 3.0

3.0

3.0 3.1

3.1

3.1 3.2 3.2

3.3

Solstice

Worse

Better

only 6% worseDefault

. 3 . 40 2 2 . 4 . 40 1 45 . 4 . . 2 48 . 142 . 3 4 .

. 43 . 40 4242 . 41 . 4041 45 . 44 . . 42 48 . 4142 . 43 44 .

Download - Scheduling Techniques for Hybrid Circuit/Packet …./mmukerje/Solstice-Conext15-Final.pdf · Scheduling Techniques for Hybrid Circuit/Packet Networks ... Circuit Switching Packet

Top Related