can we emulate local circuit switching in cloud storage?

.

Setting the Mood

• "It's time to get rid of TCP/UDP protocols in DCs"

• DCs are closed worlds, new technologies are welcome

• with bulk transfers (BigData, ...), the business value of TCP/UDPalternative is high

• circuits are an alternative to packets

M.Zhanikeev -- [email protected] -- Can we Emulate Local Circuit Switching in Cloud Storage? -- http://bit.do/marat140626 2/24...

2/24

.

Ethernet is the Best

.Ethernet.....

.

... is the cheapest and most available technology with e2esupport

• Fiber Channel (FC), SATA, etc. require expensive hardware, lowcompatibility, no e2e support

• FCoE = Ethernet, same problems, expensive hardware, no e2e support

• network virtualization is best fit for Ethernet


3/24

.

Ethernet is the Worst

.Ethernet.....

.... is the worst technology in terms of throughput• CSMA/CD is the biggest throughput limitation

◦ not in modern switches, but still major problem in wireless

• apart from CSMA/CD, contention cannot be easily resolved


4/24

.

Ethernet Contention


5/24

.

Ethernet and Contention

• whaterver you do, Ethernet L2 domains cannot avoid contention

Switch Switch

Qualitatively Identical


6/24

.

Some Contention Control• you can turn off your NIC, no TX, but some RX


7/24

.

Contention in Practice (2 flows)

20 24 28 32 36 40Transfer time in contention (s)

20

24

28

32

36

40Tr

ansf

er ti

me

by e

xclu

sive

circ

uits

(s)


8/24

.

Switches


9/24

.

Switches : Basic Facts

• cut-through versus store-and-forward• cut-through is 10 15x better

• Cisco has advanced cut-through : +bytes versus routing decision tradeoff

• store-and-forward is subjected to QoS classes◦ L3 DSCP versus L2 CoS, AF, EF, BE, SBE models


10/24

.

Switches : Modeling

C: Cut Through

Check, etc. Q: Queue

D: Drop QoS classes


11/24

.

Switches : Modeling (2)

.Model Reading..

.

Shorter paths have lower e2e delayand higher throughput

C: Cut Through

Check, etc. Q: Queue

D: Drop QoS classes

• Ta means throughput of path a

• TC > TSF

• TSF > TSQF

• TSQF > TSQD


12/24

.

Proposal


13/24

.

Proposal : Circuits

.Circuits..

.

... are emulations which allow for exclusive access to L2 domain byindividual parties

• circuits-over-packets emulation

• cut-through mode for each circuit is guaranteed

• highest possible throughput

• NOTE: will work with cheepest switches


14/24

.

Basic Math

• circuits

R = C

• packets

R = f(wCkV

)• C capacity, R throughput/rate, wweight, k parallel count, V bulk in bytes,d e2e delay, pL loss rate

• packets: w = V case is theupper bound for throughput

• packets: in reality, upper bound isnever achieved

• circuits: R = C is achievable locally(DC intranet)


15/24

.

Trivial Optimization

• special case: traffic engineering + scheduling• time shifts are optimized, bulk is known in advance

• implementation: GA, works well in small (DC-level) networks


16/24

.

Non-Trivial Optimization

Bulk size per transmission

Goodput

2 potential distributions in practice

Response curve(s)


17/24

.

Implementation


18/24

.

Implementation : 2 cases• left: book-then-send, right: separate control layer

SWITCH

NOC

Storage Node A

Storage Node B

Step 1: Book

session

Step 2: Transfer bulk

SWITCH

Storage Node A

Storage Node B

SWITCH

Bookingsegment

BulkSegment


19/24

.

Impl.: Centralized Case

SWITCH

NOC

Storage Node A

Storage Node B

Step 1: Book

session

Step 2: Transfer bulk

• same network for booking andcircuits

• inefficient but still valid/practical

• legacy-compatible,partial implementation, etc.


20/24

.

Impl.: Distributed Case

SWITCH

Storage Node A

Storage Node B

SWITCH

Bookingsegment

BulkSegment

• book on one network, send on another

• legacy-incompatible• contention-sensing possible →fully distributed models


21/24

.

Implementation : Details

• working prototype of legacy-compatible system is ready

• using for backup, transfers of VM images, etc.


22/24

.

Future FollowUps

• Goal 1: non-legacy, sensing implementation• Goal 2: optimizations in DC-size 6-10 hop networks

• Goal 3: SDN package with PICO8 switches

• Goal 4: Advanced/Curve Tradeoff Optimization models


23/24

.

That’s all, thank you ...


24/24

.

[01] myself (2014)High Availability Cloud Storage...NS研

[02] Cisco (2014)LAN Switching and Wireless, CCNA Exploration Companion GuideCisco Press

[03] Cisco (2014)Cut-Through and Store-and-Forward Ethernet Switching for Low-Latency....Cisco Press

[04] NetOptics (2014)Cut-Through Ethernet Switching: A Versatile Resource for Low Latency...White Paper

[05] Cisco (2006)QoS: DSCP Classification GuidelinesRFC4594


24/24

.

[06] Cisco (2010)A Differentiated Services Code Point (DSCP)...RFC5865

[07] open source (current)PICA8 Project for Low Latency Virtual Networkinghttp://www.pica8.com/


24/24

can we emulate local circuit switching in cloud storage?

Technology

cloud storage

local circuit

ethernet contention

storage node b step

contention switch switch

throughput packets

c packets r

contention control