cisco unified computing and virtualization: architecture, design and deployment - architecture and...

3. Need for better high availability (HA) and lower fate sharingNeed to achieve higher scalability (L2 Multipath, L3, MAC, VMs, etc.)Need to accommodate diverse workloads concurrentlyNeed to further simplify operational modelsNeed better network visibilityNeed to be prepared for:These require Cisco to address: Intel next-gen CPUs Increase feature, function and scale without increasing PCIe 3.0complexity 10G LOMsContinue to leverage/develop standards protocols to 10G-T support open and interoperable environments 40G Uplinks Increase visibility, instrumentation and manageability 100G Interconnects QSFP+ optics Evolutionary Steps prevent disruption to operational models 3

4. Denser Server Cabinets, Denser PODs & Denser X-connectsLonger cable distances, diverse connector typesServer Migration to 10Guplinks to higher density 10G or 40GEarly 40G server adoption specific workloadsHigher 10G and 40G switch density per RU & Denser 2nd switch tierFlexible L2/L3 Boundary Placement & Redundancy from 1+1 to N+1Virtual Machine & bare metal mobility: within and across DCsApplication logical isolation at scaleApplication processing closer to the wire4

5. Cisco UnifiedData Center Fabric 5

6. client-to-server client-to-serverAggregation AggregationL3ServiceServiceprocessingsrv-to-srv or vm-to-vm processingAccessAccessL2 srv-to-srv srv-to-srv or vm-to-vm srv-to-srvVirtual Access Virtual AccessL2VM-to-VM srv-to-srv or vm-to-vmsrv-to-srvVM-to-VMsrv-to-srv srv-to-srv srv-to-srv srv-to-srvSrv-to-clientSrv-to-clientsrv-to-srvTraffic Patterns Changing: More server to server traffic, and more L2 server to server traffic. Apps such as VM mobility, clustering, intra-Tier and largersubnets Client to server traffic to same subnet instances across DCs: increase of /32 from DC out Virtual Server Environments could perform a fair degree of local switching Server Roll-outs and workload movement require physical and network infrastructure coordination 2011 Cisco and/or its affiliates. All rights reserved.Cisco Confidential 6

7. Hypervisor based server virtualization and the Data Center Row 1 associated capabilities (VM Mobility, ) are changing multiple aspects of the Data Center design Where is the server now? Where is the access port? Where does the VLAN exist? Any VLAN Anywhere? How large do we need to scale Layer 2? Data Center Row 2 What are the capacity planning requirements for flexible workloads? Where are the policy boundaries with flexible workload (Security, QoS, WAN acceleration,)? 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 7

8. Server, Storage, Application and Facilities are driving Layer 2 Scalability requirementsServer Virtualization and Clusteringdriving the need for every / any VLANeverywhere based designFacilities requirements defining thenetwork topology No watt shall beleft behindVM requirements along with Data Storage growth mandating a need for more efficient and pervasive network based storage Technology changes will impact any cabling plant designMigration to 10GE as the default LoMtechnology 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 8

10. IEEE 802Evolution of Ethernet 10 GE, 40 GE, 100 GE, copper and fiberEvolution of switching DCB: Data Center Bridging 802.1BR: Bridge Port Extension INCITS/T11Evolution of Fibre ChannelFCoE (Fibre Channel over Ethernet) IBTA (Infiniband Trade Association)RoCE (RDMA over converged Ethernet), aka IBoE or RoE IETFLayer 2 Multi-Path (L2MP)TRILL (Transparent Interconnection of Lots of Links) 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 10

12. Scaling UP the Network Pod and Scaling OUT the Fabric Scaling Up of the building blocks (High Density 10G, Unified IO, FEX, Adapter-FEX, vPC, FabricPath) Scaling Out of the Fabric (FabricPath, OTV, OSPF/EIGRP/ISIS/BGP, MPLS)Scaling OUT the Fabric Scaling UP the aggregation blockVM VM VM(POD)VM VM VM #2 #3 #4VM VM VM #2 #3 #4#2 #3 #4 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 12

13. TraditionalScalable PODs Scalable FabricTopological Approach FEX and switch scaling Multipathing Strategic InvestmentsL2 / L3 10/40 GbE Leadership Fabric Scale (L2 & L3) Manageability and Programmability Enterprise, SMB Location Independence Cloud providers, large DC Enterprise, SMB, HPC/ Mobility 100s 1,000s Control Plane Intelligence100s - 10,000s10,000s - 100,000sservers per POD Servers/VMs per PODServers/VMs per POD Simplicity of Management 2011 Cisco and/or its affiliates. All rights reserved.Cisco Confidential 13

15. Network Planes of Operation The business glue of the network. Rules execution, decisionPolicy making, Service Manager and all the other components to make aPlaneproductize service. ServicesOverlay Layer 7 application flow built on the foundation of thePlaneother layers. Dependent on the other layers.Management The management plane is the logical path of all traffic related to Plane the system management of the platform. Control Its the brain of any networking platform and the technical glue of the network. The control plane is where all routing, switching,Planeother protocols and control information are exchanged The data plane receives, processes, and transmits network data Data Planebetween network elements, and represents the bulk of network traffic that passes to and through the gear. 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 15

16. Fully specifies a Port Extender (FEX Equivalent)Extended Bridge Extends ports of a switch to lower entities in a network Port Extenders are not individually managed Controllin g Their ports become ports of the controlling switchPE Bridge Cascading Port Extenders Allows one to choose the appropriate controlling switch PEPE Frame replication supported for efficient multicast / floodingBridgePE Traffic from each Extended Port is reliably segregated to an E-channel and identified by a tag containing an E-channel identifier (ECID) Does not require prior knowledge of MAC addresses; switch performs standard learning functionsServer PE Works with all devices including VEBs, VEPAs, individual VMs,ECID physical services, and devices providing transparent servicesVMvF 1W Controlling Bridge + PE = Extended Bridge Single Point of Management PE Port Extender 2011 Cisco and/or its affiliates. All rights reserved.Cisco Confidential 16

17. Distributed Modular System to the ToR, Server, and Virtual MachineOne NetworkParent Switch to Top of RackFEX ArchitectureNetworkAdministrator IEEE 802.1 BR*Consolidates network managementFEX managed as line card of parentMany applicationsFEXswitchrequireUses Pre-standard IEEE 802.1Qbhmultiple interfacesLegacy*IEEE 802.1QR Pre-Standard 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 17

18. Distributed Modular System to the ToR, Server, and Virtual MachineOne Network Parent Switch to AdapterNetworkAdministrator IEEE 802.1 BR* Adapter FEX FEX Consolidates multiple 1Gb interfaceMany applications into a single 10Gb interfacerequiremultiple interfacesExtends network into server Uses Pre-standard IEEE 802.1QbhIEEE 802.1 Qbh*Legacy Adapter FEX *IEEE 802.1QR Pre-Standard 2011 Cisco and/or its affiliates. All rights reserved.Cisco Confidential 18

19. Fabric Extender Evolution Distributed Modular System to the ToR, Server, and Virtual Machine One NetworkVirtual Same As PhysicalNetworkAdministrator IEEE 802.1 BR* FEX VM-FEXConsolidates virtual and physicalnetworkEach VM gets a dedicated port onIEEE 802.1 Qbh* IEEE 802.1 Qbh* switchUses Pre-standard IEEE 802.1QbhHypervisorVM networkmanaged byServeradministratorLegacy Adapter FEXVM-FEX*IEEE 802.1QR Pre-Standard 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 19

20. Distributed Modular System to the ToR, Server, and Virtual Machine One Network Parent Switch to Application Single Point of ManagementNetworkAdministratorFEX Architecture IEEE 802.1 BR*Manage network all Consolidates network management FEXthe way to FEX managed as line card of parentthe OS interface switchPhysical and Adapter FEXVirtual Consolidates multiple 1Gb interfaceIEEE 802.1 Qbh* IEEE 802.1 Qbh* into a single 10Gb interface Extends network into server VM-FEX Consolidates virtual and physicalHypervisor network Each VM gets a dedicated port on switchLegacy Adapter FEXVM FEX *IEEE 802.1QR Pre-Standard 2011 Cisco and/or its affiliates. All rights reserved.Cisco Confidential 20

21. 1 2 3 4 5786 SwitchNexus 5500 EthEth 1 2 345Port Extension802.1BR PE Tag 802.1BRPortExtender1Nexus 2200 (FEX)123 PE TagServer 802.1BRAdapter Port 0Port 1 Port n HypervisorNIV Capable Adapter VM VM VM VM VM VM vNICvNIC vNICvNIC vNIC1 23 45IEEE Bridge Port Extender= Cisco FEX (Fabric Extender) 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 21

22. Nexus 2000 Fabric Extender (FEX)Cisco Nexus 7000 Cisco Nexus 5500+ +Distributed High DensityEdge Switching SystemCisco Nexus 2000 FEX Cisco Nexus 2000 FEX 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 22

23. MultiChassis EtherChannel (MCEC) vPC is a Port-channeling conceptextending link aggregation to two separatephysical switches Allows the creation of resilient L2topologies based on Link Aggregation.Physical Topology Logical Topology Eliminates the need for STP in theVirtual Port Channel access-distribution L2 Provides increased bandwidth Si Si All links are actively forwarding vPC maintains independent control planeNon-vPCvPCIncreased BW with vPC 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 23

24. Co-existence of LAN and SAN LAN and SAN utilize different High Availability Models SAN is dual fabric, LAN is fully meshed fabric vPC enables both architectures at the edge (single device models not acceptable to SAN customers) WAN Core FCCore CoreL3AggregationL2 Access Edge 2011 Cisco and/or its affiliates. All rights reserved.Cisco Confidential 24

26. Cisco Nexus 5x00 and 2200represent a virtual accessswitch POD VPC pair Nexus 7000 at AggregationLayer NO Loop Nexus 5x00/2200VirtualizedAccess Switch PODs ...NO STP 2011 Cisco and/or its affiliates. All rights reserved.Cisco Confidential 26

27. Nexus 7000 vPC+Unified ComputingSystem (UCS)Nexus 5000 / 5500 + 2200Virtual Access Switch PODNexus 7000 + 2200Virtual Blade SwitchingVirtual Access (VBS) Switch POD 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 27

29. Large Scale Web 2.0 Environments L31. Soft L3 on access - per ToR VLANs L32. East-west traffic: 2 tiers large scale L3 3. Limited VLAN extension overlays (ex.: OTV) L2 L3 Enterprise Environments1.VPC / STP used for L2 restricted VLAN Scale L3 L2 2.Pod traffic: 2 tiers limited scale3.Cross-pod East-West traffic = 3 tiers L2 L3 L2 Large Scale SPDC: Hosting & Cloud1. L2 on access and aggregation L22. VLAN Scale limited high fate sharing3. East-west traffic: 2 or 3 tier L2 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 29

30. Data Center Fabric Topologies & Attributes - Trending L3 Cloud Large Scale Web 2.0 Environments1. L3 on access - per ToR VLANs L3 2. Migration to 10G: 3 tiers very large scale3. Broad VLAN extension through overlays L3 L2 L3 FabricsCommonalities between Enterprise & SPDC1. L2MP (FabricPath) used for L2 increase VLAN spread L3 L3 Cloud 2. 2-tier east to west traffic L3 3. N-way tier 2 (spine) L2Differences between Enterprise & SPDC L21. Host Density L3/L22. VLAN Scale virtualization scaleFabrics3. Public-cloud vs private cloud 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 30

31. The protocol choices allow like topologies to be buildEqual Cost Multi Path (ECMP) over L2 or L3Plug and Play Nature of L2 ProtocolsRedundancy, stability and scale of Layer 3 ProtocolsHigh Availability models become similar for L2 and L3: N+1 redundancyDifferent Environments Have:A preferred placement for the L2/L3 boundaryA High-Availability, Scale, and functional targetsThese Converge given the flexibility offered by the forthcoming protocolsChoice of L2 or L3 Protocols does not prevent redundancy and multi-pathing requirementsLocation of L2/L3 boundary does not prevent adjacency or redundancy optionsL2/L3 Boundary becomes less relevantCLOS Topologies dominate new implementationsHigh Availability models shiftServer Edge becomes more intelligentData Center Fabric becomes more scalable 2011 Cisco and/or its affiliates. All rights reserved.Cisco Confidential 31

32. L3 L3/L2 L3/L2L2L2 East-West traffic Fate Sharing Domain Larger POD East-West Traffic Fate Sharing Domain STP has been the protocol of choice N+1 redundancy 1+1 redundancy limited forwarding paths IS-IS is the protocol of choice Replicated Stateful Services per PodBroad forwarding paths Broader Adjacency Support East-West across L3 boundaries Service Insertion is not replicated OSPF/EIGRP are protocols of choice N+1 redundancy Broad forwarding Paths Same number of physical boxes and links North-South traffic Protocol behavior is L3-like OSPF/EIGRP are protocols of choiceMulti-pathing over L2 and L3 N+1 redundancy Broad forwarding paths More flexible L2 adjacency, better scale capacity Better latency consistency within POD 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 32

34. Focused on Stability FeaturesN Network port (Bridge Assurance)E Edge port- Normal port type Data CenterB BPDUguard Core R RootguardL LoopguardF Global BPDU filter HSRP HSRP ACTIVE STANDBY Layer 3 AggregationN N Backup Root RootLayer 2 (STP + Bridge Assurance)N N N -N N N -R R R R R R R RLayer 2 (STP + BA + Rootguard)N N AccessNN N NL L E F E FE F E F E F B BB B BLayer 2 (STP + BPDUguard) 2011 Cisco and/or its affiliates. All rights reserved.Cisco Confidential 34

35. Focused on Scalability Features, with a pair of Aggregation boxes. N Network port (Bridge Assurance) E Edge port - Normal port typeData CenterB BPDUguardCore R Rootguard L Loopguard F Global BPDU filterVPCHSRP domain HSRPACTIVEACTIVELayer 3AggregationNN RootRoot Layer 2 (STP + Bridge Assurance)- - - - -- - - R R R RR R R RLayer 2 (STP + Rootguard)-Access-- L EEEEE FFFFF BBBBBLayer 2 (STP + BPDUguard) 2011 Cisco and/or its affiliates. All rights reserved.Cisco Confidential 35

36. MAC addresses encode no location or network hierarchy Default forwarding behavior in bridged network is flood MAC filtering database limits scope of flooding Ultimately, may not scale well as every switch learns every MAC MAC TableMAC TableAALayer 2Domain MAC Table MAC TableMAC Table A MAC TableAA A 2011 Cisco and/or its affiliates. All rights reserved.Cisco Confidential 36

37. MAC v.s. IP Network Address 10.0.0.0/240011.1111.111110.0.0.10 /24Non-hierarchical Host Address Address 10.0.0.10 0011.1111.1111 0011.1111.111110.0.0.0/16 20.0.0.0/16 0011.1111.1111 10.0.0.0/24 20.0.0.0/24 0011.1111.11110011.1111.111110.0.0.10 20.0.0.20 L2 Forwarding (Bridging)L3 Forwarding (Routing) Data-plane learning Control-plane learning Flat address space and forwarding Hierarchical address space andtable (MAC everywhere !!!)forwarding Flooding required for unknown unicast Only forwarding to destinationdestination addresses with matching routes in the Destination MACs need to be knowntablefor all switches in the same network toFlooding is isolated within subnetsavoid flooding No dependence on data-plane formaintaining forwarding table 2011 Cisco and/or its affiliates. All rights reserved.Cisco Confidential 37

38. What Can Be Improved? Network Address Scheme: FlatHierarchical Additional header is required to allow L2 Routing instead of Bridging. Switch ID comes to the picture. Provide additional loop-prevention mechanism like TTL Address Learning: Data Plane Control Plane Eliminate the needs to program all MACs on every switches to avoid flooding Control Plane: Distance-Vector Link-State Improve scalability, minimize convergence time, and allow multipathing inherently The ultimate solution needs to take both control and data plane intoconsideration this time!!! 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 38

39. Cisco FabricPathData Plane InnovationControl Plane InnovationFabricPath encapsulation Plug-n-Play Layer 2 IS-ISConversation LearningSupport unicast and multicastRouting, not bridgingFast, efficient, and scalableBuilt-in loop-mitigationEqual Cost Multipathing Time-to-Live (TTL) (ECMP) RPF Check VLAN and Multicast Pruning Cisco NX-OS Cisco Nexus Platform 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 39

40. Multi-Domain SilosFabricPath Any App, Anywhere!FabricWeb ServersApp Servers New AppsWeb ServersApp ServersSilo 1 Silo 2Silo 3 New Apps Benefits server team by providing a network Fabric that looks like a single switch Breaks down silos, permits workload mobility, provides maximum flexibility Lowers OPEX by simplifying server team operation Reduces dependency on/interaction with network team 2011 Cisco and/or its affiliates. All rights reserved.Cisco Confidential 40

41. Externally, a Fabric looks like a single switch Internally, a protocol adds Fabric-wide intelligence and ties the elements together. This protocol provides in a plug-and-play fashion:Optimal, low latency connectivity any to anyHigh bandwidth, high resiliencyOpen management and troubleshooting Cisco FabricPath provides additional capabilities in term of scalability and L3 integrationFabricPath FabricPath 2011 Cisco and/or its affiliates. All rights reserved.Cisco Confidential 41

42. Enabling Network Fabrics FabricPath Connect a group of switches using an arbitrary topology With a simple CLI, aggregate them into a Fabric: N7K(config)# interface ethernet 1/1 N7K(config-if)# switchport mode fabricpath An open protocol based on L3 technology provides Fabric- wide intelligence and ties the elements together 2011 Cisco and/or its affiliates. All rights reserved.Cisco Confidential 42

43. Example 1: Classical POD Migration Q: Why migrate traditional Access/ Aggregation building block to FabricPath? A: No STP No STP sync, no topology changes, no blocked ports, no risk of loops Simple configuration Total flexibility in design and cabling Enables organic bandwidth growth Grow where and whenever needed with minimal impact 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 43

44. Q: How to interconnect DC PODs in order to have VLANs anywhere? A: Provide server/host connection to any edge port in network, regardless of physical location Physical/rack/distribution pair location of host irrelevant with respect to IP subnet and Layer 2 adjacency with other hosts Gateway placement options include GLBP, MHSRP and leaf-attached gateways, so far.POD 1 POD 2 POD 3 PODS 1-3 VLANs 100-199VLANs 200-299VLANs 300-399 VLANs 100-399 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 44

45. Cisco FabricPathSpanning-Tree vPC FabricPath16SwitchesActive PathsSingleDual 16 WayPodUp to 10 TbpsUp to 20 Tbps Up to 160 TbpsBandwidth 2011 Cisco and/or its affiliates. All rights reserved.Cisco Confidential 45

47. Terminology Interface connected to another FabricPath deviceSends/receives traffic with FabricPath headerDoes not run spanning treeDoes not perform MAC learning!Exchanges topology info through L2 ISIS adjacencyForwarding based on Switch ID TableFP Core Ports S10 S20 S30S40 Spine Switch FabricPath (FP)S100 S200 S300 Leaf Switch1/1 1/2 Classical Ethernet (CE)A B CE Edge PortsInterface connected to traditional network deviceSends/receives traffic in standard 802.3 Ethernet frameformatParticipates in STP domainForwarding based on MAC table 2011 Cisco and/or its affiliates. All rights reserved.Cisco Confidential 47

48. A New Data Plane The association MAC address/Switch ID is maintained at the edge S10 S20S30S40Switch ID space:S300: FabricPathRouting Routing Tabledecisions areA B S100 S300made based on SwitchIFthe FabricPathrouting tableFabricPath (FP) S100 S200 S300 S100L1, L2, L3, L4MAC address1/11/2S300: CE MACspace: Address TableSwitching based Classical Ethernet (CE) MACIFon MAC address ABB1/2tables A S100 Traffic is encapsulated across the Fabric 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 48

49. Conversational MAC Learning S10 S20 S30 S40A B S100M FabricPath Lookup B: HitS100S200 S300 Learn source ALookup B: MissFloodLookup B: Miss Dont learn S100: CE MAC1/1S200: CE MAC 1/2S300: CE MAC Address TableAddress Table Address TableMAC IFMAC IF MACIF A B A 1/1B 1/2AS100 Classical Ethernet 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 49

50. Conversational MAC Learning S10 S20S30 S40S300: FabricPath Routing Table B A S300 S100Lookup A: HitLookup A: Hit SwitchIF Learn source B FabricPath Send to S100S100 S200 S300 S100 L1, L2, L3, L4 S100: CE MAC1/1 S200: CE MAC 1/2S300: CE MAC Address Table Address Table Address TableMAC IF MACIF MAC IF AB A1/1 B1/2 B S300 A S100 Classical Ethernet Conversational Learning 2011 Cisco and/or its affiliates. All rights reserved.Cisco Confidential 50

51. Its a Routed Network Describes shortest (best) paths to each Switch IDbased on link metrics Equal-cost paths supported between FabricPathswitchesS10 S20 S30S40FabricPathRouting Table onS100 Switch IFOne best path S10L1to S10 (via L1)S20L2S30L3S40L4 S200 L1, L2, L3, L4 S100 S200FabricPathS300 Four equal-cost S300 L1, L2, L3, L4paths to S300 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 51

52. (1) Broadcast ARP RequestRoot for Root forMulti-destination Tree 1 Tree 2Trees on Switch 10S10S20 S30 S40 Tree4IFFtag 1 po100,po200,po300po3002 po100DAFFpo100 po200Ftag1SA100.0.12Multidestination DAFFDMACFFFtag1Trees on Switch 100po10 po20 po30 po40SMACA po10 po20 po30 po40 SA100.0.12Tree 3IF S300Payload DMACFFS100 Broadcast 1 po10 S200 SMACA 2po10,po20,po30,po40Multidestination Payload Trees on Switch 300 5 6 FabricPath e1/13Tree IF e2/29 Payload MAC Table on S100DMACFF SMACASMACAFtag 1po10,po20,po30,po40MAC IF/SID DMACFF2 po40Ae1/13 (local) 2PayloadMAC A1 MAC B FabricPath MAC Table on S200MACIF/SIDDont learn MACs fromLearn MACs of directly-connected framesflooddevices unconditionally 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 52

53. MAC Address Table after the first ARP frame S100:S100# sh mac address-table dynamicLegend:* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MACage - seconds since last seen,+ - primary entry using vPC Peer-LinkVLAN MAC AddressTypeage Secure NTFY Ports/SWID.SSID.LID---------+-----------------+--------+---------+------+----+------------------* 10 0000.0000.000a dynamic 0FF Eth1/13MAC A learned aslocal entry on e1/13 S10 (and S20, S30, S40, S200):S10# sh mac address-table dynamicLegend:* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MACage - seconds since last seen,+ - primary entry using vPC Peer-LinkVLAN MAC AddressTypeage Secure NTFY Ports/SWID.SSID.LID---------+-----------------+--------+---------+------+----+------------------ S300: MAC A notS300# sh mac address-table dynamic learned on other switchesLegend:* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MACage - seconds since last seen,+ - primary entry using vPC Peer-LinkVLAN MAC AddressTypeage Secure NTFY Ports/SWID.SSID.LID---------+-----------------+--------+---------+------+----+------------------ 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 53 53

54. (2) Broadcast ARP Reply Root forRoot for MultidestinationTree 1Tree 2 S10S20S30S40 Trees on Switch 1010 TreeIF Ftag 1po100,po200,po300 po3002po100 DAMC1 Ftag1po100 po200 SA300.0.64DAMC1Ftag1 DMACAMultidestinationSA300.0.64Trees on Switch 100SMACB po10 po20 po30po40DMACApo10 po20 po30 PayloadTree11IFpo40 S300 SMACBFtag 1 po10S200MultidestinationPayload 2 po10,po20,po30,po40Trees on Switch 3009 Tree IF7FabricPath e1/13MAC Table on S100Payload Unknown 1 po10,po20,po30,po40 e2/29DMACA 2po40SMACB MAC IF/SIDSMACBPayload Ae1/13 (local) 12 DMACA MAC AFabricPath B300.0.64 (remote) MAC Table on S300 MAC B 8 MAC MAC IF/SIDIF/SIDA MISS Be2/29 (local) If DMAC is known, then learn remote MAC *MC1 = 01:0f:ff:c1:01:c0 2011 Cisco and/or its affiliates. All rights reserved.Cisco Confidential 54

55. MAC Address Table after the first ARP frame S100: S100# sh mac address-table dynamic Legend:* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MACage - seconds since last seen,+ - primary entry using vPC Peer-LinkVLANMAC Address Typeage Secure NTFY Ports/SWID.SSID.LID ---------+-----------------+--------+---------+------+----+------------------ * 100000.0000.000a dynamic 90 F F Eth1/1310 0000.0000.000b dynamic 60 F F 300.0.64 S100 learns MAC Bas remote entryreached through S100#S300 S300: S300# sh mac address-table dynamic Legend:* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MACage - seconds since last seen,+ - primary entry using vPC Peer-LinkVLANMAC Address Typeage Secure NTFY Ports/SWID.SSID.LID ---------+-----------------+--------+---------+------+----+------------------ * 100000.0000.000b dynamic 0F F Eth2/29 MAC B learned as local entry on S300# e2/29 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 55 55

56. Unicast Data - RoutedFabricPath RoutingTable on S30 S10S20S30S40 SwitchIF S300 S300 po300 16po300 DA300.0.64DA300.0.64 FabricPath Routing Ftag1 Ftag1 Table on S100 SA100.0.12SA100.0.12 SwitchIF DMACBDMACB S10 po10 SMACASMACA po10 po20 po30 po40 S20 po20po10 po20 po30 PayloadPayloadHashpo40 S30 po30 S300 S40 po40 S200 FabricPath RoutingS100po10, po20,Table on S300S200po30, po401715 SwitchIFpo10, po20,e1/13S300 PayloadS300 po30, po40e2/29S300 S300Use LID (64) SMACA DMACB FabricPathDMACB MAC Table on S100 SMACAFabricPath MAC AMACIF/SIDPayload MAC Table on S300 MAC B Ae1/13 (local) 14 13MACIF/SID 18A S100.0.12 (remote) BB 300.0.64 (remote)If DMAC is known, thenlearn remote MACBe2/29 (local) 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 56

57. Unicast forwarding S100:S100# sh mac address-table dynamicLegend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vPC Peer-Link VLAN MAC Address Typeage Secure NTFY Ports/SWID.SSID.LID---------+-----------------+--------+---------+------+----+------------------* 10 0000.0000.000a dynamic 90 F F Eth1/1310 0000.0000.000b dynamic 60 F F 300.0.64S100# S300:S300# sh mac address-table dynamicLegend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vPC Peer-Link VLAN MAC Address Typeage Secure NTFY Ports/SWID.SSID.LID---------+-----------------+--------+---------+------+----+------------------ S100 learns MAC A10 0000.0000.000a dynamic 30 F F 100.0.12 as remote entryreached through S100* 10 0000.0000.000b dynamic 90 F F Eth2/29S300# 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 57 57

58. Unicast ForwardingS100# sh fabricpath routeFabricPath Unicast Route Tablea/b/c denotes ftag/switch-id/subswitch-id[x/y] denotes [admin distance/metric] Topology (ftag),ftag 0 is local ftagSwitch ID, Sub-subswitch-id 0 is default subswitch-idSwitch ID AdministrativeFabricPath Unicast Route Table for Topology-Default distance, routing0/100/0, number of next-hops: 0metricvia ---- , [60/0], 0 day/s 04:43:51, local1/10/0, number of next-hops: 1via Po10, [115/20], 0 day/s 02:24:02, isis_fabricpath-defaultRoute age1/20/0, number of next-hops: 1via Po20, [115/20], 0 day/s 04:43:25, isis_fabricpath-default1/30/0, number of next-hops: 1via Po30, [115/20], 0 day/s 04:43:25, isis_fabricpath-default1/40/0, number of next-hops: 1 Client protocolvia Po40, [115/20], 0 day/s 04:43:25, isis_fabricpath-default1/200/0, number of next-hops: 4via Po10, [115/40], 0 day/s 02:24:02, isis_fabricpath-defaultvia Po20, [115/40], 0 day/s 04:43:06, isis_fabricpath-defaultNext-hopvia Po30, [115/40], 0 day/s 04:43:06, isis_fabricpath-default interface(s)FabricPathvia Po40, [115/40], 0 day/s 04:43:06, isis_fabricpath-default1/300/0, number of next-hops: 4S10S20 S30S40via Po10, [115/40], 0 day/s 02:24:02, isis_fabricpath-defaultvia Po20, [115/40], 0 day/s 04:43:25, isis_fabricpath-defaultvia Po30, [115/40], 0 day/s 04:43:25, isis_fabricpath-defaultvia Po40, [115/40], 0 day/s 04:43:25, isis_fabricpath-defaultS100# po10po20po30 po40 S100 S200 S300 AB C 2011 Cisco and/or its affiliates. All rights reserved.Cisco Confidential 5858

59. S3 FabricPath CE L1L2Introducing vPC+F1VPC+ F1S1F1F1S2 Allows dual-homed connections from edge ports intoFabricPath domain with active/active forwardingF1F1 Can also provide active/active HSRP Configuration virtually identical to standard VPCPhysicalHost A Supported on both N7K (F1/F2) and 5500 VPC+ peer switches share a virtual FabricPath switch ID LogicalS3 Host AS4L1,L2 MAC addresses behind VPC+ port-channels appear asconnected to the virtual switch, not the VPC+ physical L1L2peer switchesF1VPC+ F1 Allows load-balancing within FabricPath domain toward theVPC+ virtual switch S1 F1 F1S2 VPC+ requires F1/F2 modules on N7K with FabricPath F1F1enabled in the VDC or FabricPath enabled on the Nexus5500.S4Virtual Switch 4 becomes egress switch for Host A in FabricPath domain Host A 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 59

60. Driven by multiple vendors, including CiscoTRILL is now an IETF proposed standardFabricPath will provide a TRILL mode with a software upgrade (Cisco hardware is alreadyTRILL capable)Cisco is pushing FabricPath specific enhancements to TRILLFabricPathTRILLFrame routing YesYes(ECMP, TTL, RPFC etc )vPC+YesNoFHRP active/activeYesNoMultiple topologies YesNoConversational learning YesNoInter-switch links Point-to-point only Point-to-point OR sharedhttp://datatracker.ietf.org/wg/trill/ 2011 Cisco and/or its affiliates. All rights reserved.Cisco Confidential 60

61. HSRPHSRPL3 B L3 B Active/Active Active/Standby FabricPathG1 G2 TRILLG1 G2 MultipathingMultipathinga1a2 a5 a6a1a2 a5a6VPC+ TRILLActive/ActiveActive/Standby ACA C End-to-end multipathing (L2 edge, Fabric, L3 edge) provides resiliency and fast convergence 2011 Cisco and/or its affiliates. All rights reserved.Cisco Confidential 61