a look inside google’s data center networks

Download A Look Inside Google’s Data Center Networks

Post on 04-Aug-2015

33.733 views

Category:

Technology

5 download

Embed Size (px)

TRANSCRIPT

1. 2015629 A Look Inside Googles Data Center Networks Amin Vahdat, A Look Inside Googles Data Center Network, 2015 Open Network Summit Keynote. http://www.theplatform.net/2015/06/19/inside-a-decade- of-google-homegrown-datacenter-networks/ 2. Googles warehouse-scale networking infrastructure 2 B4 [SIGCOMM 2013] Andromeda [ONS 2014] Firehose -> Watchtower -> Saturn -> Jupiter [ONS 2015, SIGCOMM 2015] 3. B4: Globally-deployed SDN WAN 3 Google Confidential and Proprietary Google's OpenFlow WAN Google Confidential and Proprietary G-Scale Network Hardware Built from merchant silicon 100s of ports of nonblocking 10GE OpenFlow support Open source routing stacks for BGP, ISIS Does not have all features No support for AppleTalk... Multiple chassis per site Fault tolerance Scale to multiple Tbps 30% 70% OpenFlow@Google, 2012 Open Network Summit 4. B4: Globally-deployed SDN WAN OpenFlowOGF NSI 4 5. Andromeda network virtualization Goal: the raw performance of the underlying network while simultaneously exposing NFV 5 http://googlecloudplatform.blogspot.jp/2014/04/enter-andromeda- zone-google-cloud-platforms-latest-networking-stack.html 6. Trac generated by servers in Google DCs 6 7. Motivation Traditional network architectures could not keep up with bandwidth demands in the DC Operational complexity of box-centric deployment Inspired by server and storage scale out, employed three principles to redesign DCN Clos Technologies Merchant Silicon Centralized Control 7 Amdahls lesser known law (late 1960s): 1Mbit/sec of IO for every 1Mhz of computation in parallel computing 1 server (64*2.5 Ghz) : 100 Gb/sec 50k servers : 5 Pb/sec!! 8. Google DCN Over Time 8 Datacenter generation Year Merchant Silicon ToR configuration Aggregation block Spine block Fabric speed Host speed Aggregate bandwidth Four-Post CRs 2004 48x1G - - 10G 1G 2Tb/sec Firehose 1.0 2005 8x10G, 4x10G 2x10G up, 24x1G down 2x32x10G 32x10G 10G 1G 10Tb/sec Firehost 1.1 2006 8x10G 4x10G up, 48x1G down 64x10G 32x10G 10G 1G 10Tb/sec Watchtower 2008 16x10G 4x10G up, 48x1G down 4x128x10G 128x10G 10G 1G 82Tb/sec Saturn 2009 24x10G 24x10G 4x288x10G 288x10G 10G 10G 207Tb/sec Jupiter 2012 16x40G 16x40G 8x128x40G 128x40G 10G/40G 10G/40G 1.3Pb/sec (- 2004) (2005 -) 9. 9 Datacenter generation Year Merchant Silicon ToR configuration Aggregation block Spine block Fabric speed Host speed Aggregate bandwidth Firehose 1.0 2005 8x10G, 4x10G 2x10G up, 24x1G down 2x32x10G 32x10G 10G 1G 10Tb/sec Firehost 1.1 2006 8x10G 4x10G up, 48x1G down 64x10G 32x10G 10G 1G 10Tb/sec 10. 10 Datacenter generation Year Merchant Silicon ToR configuration Aggregation block Spine block Fabric speed Host speed Aggregate bandwidth Watchtower 2008 16x10G 4x10G up, 48x1G down 4x128x10G 128x10G 10G 1G 82Tb/sec (four edge aggr. switches) 11. 11 Datacenter generation Year Merchant Silicon ToR configuration Aggregation block Spine block Fabric speed Host speed Aggregate bandwidth Saturn 2009 24x10G 24x10G 4x288x10G 288x10G 10G 10G 207Tb/sec 12. 12 Datacenter generation Year Merchant Silicon ToR configuration Aggregation block Spine block Fabric speed Host speed Aggregate bandwidth Jupiter 2012 16x40G 16x40G 8x128x40G 128x40G 10G/40G 10G/40G 1.3Pb/sec 13. Jupiter Superstack 13 14. Facebook 6-pack open hardware modular switch 14 40Gbps x 64 ports Dual-stage clos network Linux-based Operating System (FBOSS) 15. 25/50 Gigabit Ethernet 25 Gigabit Ethernet Consortium Arista, Broadcom, Google, Mellanox, Microsoft http://25gethernet.org/ 2550100 Alliance http://www.2550100.com/ 15