the data center network evolution
TRANSCRIPT
Journey to the Programmable FabricThe Data Center Network Evolution
Robert ZalobinskiTechnical Solutions [email protected]
In partnership with:
Cisco Confidential 2© 2013-2014 Cisco and/or its affiliates. All rights reserved.
• Basics of SDN and Overlay Networks
• Application Centric Infrastructure (ACI)
• Virtual Topology System (VTS 2.0)
• Unified Open NX-OS
Cisco Confidential 3© 2013-2014 Cisco and/or its affiliates. All rights reserved.
What is SDN
Software defined networking (SDN) is an approach to building computer networks that separates and abstracts elements of these systems
In other words…
In the SDN paradigm, not all processing happens inside the same device
Cisco Confidential 4© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Current Network ArchitectureTightly coupled Control and Data Planes
• One Control Plane per Device
• Each Device Managed Individually
• All Command Line Managed
Routing protocols (i.e. OSPF, IS-IS, BGP), Spanning Tree, SYSLOG, AAA
(Authentication Authorization Accounting), NDE (Netflow Data Export), CLI
(Command Line interface), SNMP
Layer 2 switching, Layer 3 (IPv4 | IPv6) switching, MPLS forwarding, VRF
Forwarding, QOS (Quality of Service) Marking, Classification, Policing, Netflow flow collection, Security Access Control
Lists
cpu
asic
Cisco Confidential 5© 2013-2014 Cisco and/or its affiliates. All rights reserved.
The Promise of SDN
Overlay Protocol
Physical Network
10001101000110101
Control & Data PlaneDecoupled
NetworkVirtualization
DirectProgrammability
Centralized ManagementSimplification
Agility Programmatically ConfiguredDynamic
Automated
Cisco Confidential 6© 2013-2014 Cisco and/or its affiliates. All rights reserved.
SDN Programming Models
Applications Control
DataAPI
ControllerAPI API Data
ControllerAPI API Contr
ol
Data
vSwitchAPI OverlayProtocol
Control
DataOverlay
Controller
Cisco Confidential 8© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Types of Overlay Edge Devices
• Virtual end-points only
• Single admin domain
• VXLAN, NVGRE, STT
• Physical and Virtual
• Resiliency + Scale
• Cross-organizations/Federation
• Open Standards
Network Overlays Integrated OverlaysHost Overlays
• Router/switch end-points
• Protocols for resiliency/loops
• Traditional VPNs
• OTV, VXLAN, VPLS, LISP
App
OS
App
OS
Virtual Physical
Fabric DB
VM
OS
VM
OS
Virtual Virtual
VM
OS
VM
OS
Physical Physical
Cisco Confidential 9© 2013-2014 Cisco and/or its affiliates. All rights reserved.
VXLAN Overview
Outer MACDA
Outer MACSA
Outer 802.1Q
Outer IP DA
Outer IP SA
Outer UDP
VXLAN ID
(24 bits)
Inner MAC DA
InnerMACSA
Optional Inner
802.1Q
Original Ethernet Payload
CRC
VXLAN Encapsulation Original Ethernet Frame
CRC
Data
Plane
16 M Segments
Control
Informatio
n
Tunnel Endpoints DiscoveryHost Reachability Information
• Mac Address• IP address
Draft Data Plane Multicast based flood and learn
Cisco Confidential 10© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Eth Eth Eth
vEth vEth vEth vEth vEth vEth
Overlay Network Communications - VTEP
VXLAN utilizes a VTEP:• Virtual Tunnel End Point• IP address assigned• Layer-3 Transportable• IP/UDP Packets
10.10.10.101 10.10.10.211 172.18.22.12
Inter VXLAN communications
VTEP VTEP VTEP
Cisco Confidential 11© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Eth Eth Eth
vEth vEth vEth vEth vEth vEth
Overlay Network Communications
VXLAN 55110
VXLAN 45235
16m VXLANs
VXLAN requires a network gateway function:• VXLAN to VLAN Bridge• VXLAN to VLAN Router• VXLAN to VXLAN Router
VLANs
Cisco Confidential 12© 2013-2014 Cisco and/or its affiliates. All rights reserved.
VXLAN54210
VXLAN Gateway Functions
VXLAN55110VXLAN45235 VLAN 235
VLAN 110
VXLAN55110 VXLAN45235
VXLAN55110 VLAN 235
VXLAN to VLAN Bridging (L2 Gateway)
VXLAN-to-VXLAN Routing (L3 Gateway)
VXLAN-to-VLAN Routing (L3 Gateway)
VXLAN45235
Cisco Confidential 13© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Programmable NetworkProgrammable FabricApplication Centric Infrastructure
DB DB
Web Web App Web App
VxLAN-BGP EVPN standard-based
3rd party controller support
Modern NX-OS with enhanced NX-APIs
Automation Ecosystem (Puppet, Chef, Ansible etc.)
Common NX-API across N2K-N9K
Turnkey integrated solution with security, centralized management,
compliance and scale
Automated application centric-policy model with embedded security
Broad and deep ecosystem
Cisco SDN: Providing Choice in Automation and Programmability
Mass Market (commercial, enterprises, public sector)
Service Providers Mega Scale Datacenters
VTS for software overlay provisioning and management
across N2K-N9K
14© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Application Centric Infrastructure (ACI)
Cisco Confidential 15© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Two Types of Languages
Infrastructure Applications
HumanTranslator
• Application Tier Policy and Dependencies
• Security Requirements• Service Level Agreement• Application Performance• Compliance• Geo Dependencies
• VLAN• IP Address• Subnets• Firewalls • Quality of Service • Load Balancer• Access Lists
Cisco Confidential 16©2014 Cisco and/or its affiliates. All rights reserved.
Introducing: Application Centric Infrastructure
Apps + Infrastructure
Physical + VirtualOpen + Secure
On-Premises + Cloud
Application Oriented Policy = Operational Simplicity
Cisco Confidential 17© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Application Centric Infrastructure Components
Fabric
Centralized Policy ManagementOpen APIs, Open Source, Open Standards
Policy ControllerApplication Network Profile APIC
End Points
Physical Networking
Nexus 2K
Nexus 7K
Hypervisors and Virtual Networking
Compute L4–L7Services
Storage Multi DC WAN and Cloud
Integrated WAN Edge
VirtualPhysical
NorthboundManagement
IntegrationPartner Ecosystem
Automation
OVM
HypervisorManagement Monitoring
SystemsManagement
OrchestrationFramework
Cisco Confidential 18© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Typical Three Tier Application
Web Servers
Firewall
Server Load Balancer
Access Switch
Server
vSwitch
Firewall
Access Switch
App Servers
DatabaseServer
Application Requirements tightly coupled to the Network
Port Group, VLAN, IP Address, IP Mask
Interface, Trunk, VLAN, IP Subnets
Interface, Trunk, VLAN, IP Subnets
• Network Connectivity• Security Policies• Quality of Service• Layer 4 – 7 Application
Services• Storage Policies• Compute Policies• Hypervisor Policies
Cisco Confidential 19© 2013-2014 Cisco and/or its affiliates. All rights reserved.
The Policy-based Datacenter
IP Fabric
• Single APIC Controller:• End-to-end Application
Profile• ACI IP Fabric encompasses
o Infrastructureo Physicalo Virtualo Services
• ANP Profile pushed to all components
• Full Workload Mobility, Replication and Instantiation Application Network Profile
Web Servers App Servers Database Server
Cisco Confidential 20© 2013-2014 Cisco and/or its affiliates. All rights reserved.
ACI Benefit: Deep Telemetry — Application and Tenant
APIC
APP
TEN
AN
T
TenantTenant 1 Tenant 2
Tenant 3 Tenant 4
Cisco Confidential 21© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Application Approach To Networking
F/W DB DBDecouple Policy from Infrastructure
Simple & Scalable Stateless Infrastructure Optimized Forwarding & Mobility
Abstracted Policies for definition of Applications & Connectivity
Open REST APIsCentralized ManagementOpen Source APIC
Application Network Profile
F/W F/W F/W
STORAGE STORAGE
WEB DBAPP
Highest Performance & ReliabilityLowest Power Consumption
22© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Virtual Topology System (VTS)
Cisco Confidential 23© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Programmable FabricNX-API, VXLAN BGP EVPN Fabric, and Virtual Topology System (VTS)
Operations / Programmability
& Automation
Automated DCI / WAN
VM
OS
VM
OS
NX-API
Physical Virtual DCI/WAN
Bare Metal Virtualized
BGP-EVPN VXLAN Fabric
VTS
VTS for overlay provisioning and management across Nexus 2000 – Nexus 9000 (2H 2015)
Cisco Confidential 24© 2013-2014 Cisco and/or its affiliates. All rights reserved.
vCenter
REST API
VTS
GUI
Across Nexus PortfolioNexus 2K – 9K
Programmable Fabric
AutomatedSeamless integration with Orchestrators
Overlay provisioning and DCI/WAN integration
Scalable VXLAN ManagementMP-BGP EVPN control plane
High performance virtual forwarding
Open and ProgrammableREST Northbound APIs
Multi-protocol and Multi-hypervisor support
Virtual Topology System (VTS) Overlay Provisioning & Management System
Flexible OverlaysPhysical and virtual overlays
Bare-metal and Virtualized workloads
Cisco Confidential 25© 2013-2014 Cisco and/or its affiliates. All rights reserved.
VXLAN as Data Center Overlay technology
L2 L3 L4VTEP
Local LAN Local LAN Local LAN Local LAN
IP Transport Network
VTEP VTEP VTEP
VXLAN VNI
LAN Segment
Underlay Network:• IP routing – proven, stable, scalable• ECMP – utilize all available network paths
Overlay Network:• Standards-based overlay• Layer-2 extensibility and mobility• Expanded Layer-2 name space • Scalable network domain• Multi-Tenancy
Modes of Operation:• Multicast based flood and learn (No control plane)• BGP EVPN (BGP control plane with MP-BGP Extensions)
Cisco Confidential 26© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Advantages with EVPN Control Plane
Industry standard protocol for multi-vendor support
Built-in Multi tenancy support
Truly scalable with protocol-driven control plane architecture
Fast convergence upon network failures and host movements
Minimize flooding through ARP suppression
Security through VTEP peer-authenticationAdv
anta
ges
of E
VP
N
Con
trol P
lane
Cisco Confidential 28© 2013-2014 Cisco and/or its affiliates. All rights reserved.
VTF
Cisco Network Services Orchestrator VMware vCenter GUI
DVS
Unified Information Model (REST API)
YANG CLI NX-API BGP-EVPN
Virtual Topology System
Service and Infrastructure PolicyInventoryDatabaseResource Management
Policy PlaneC
ontrolPlaneIOS XRvDevice Management
Control Plane FederationMP-BGP
Cisco Nexus 2000, 3000, 5000, and 7000 Series
Cisco Nexus 9000 Series Cisco ASR 9000 Series
Virtual Compute Environment
VTS Architecture
Cisco Confidential 29© 2013-2014 Cisco and/or its affiliates. All rights reserved.
VTS Architecture
Cisco VTS
ToR ToR
Spine Spine
ToR
Hypervisor
VM
x86 Server
Hypervisor
VM
x86 Server
VTF VTF
Hypervisor
VMVM
x86 Server
REST API
DCI
NX-API, CLI, YANG
VTEP
VTEP VTEP
Border Leaf VTEP
VMware vCenter
Virtual Topology System
Service and Infrastructure PolicyInventoryDatabaseResource Management
Policy PlaneC
ontrolPlaneIOS XRvDevice Management
Cisco Confidential 30© 2013-2014 Cisco and/or its affiliates. All rights reserved.
VTS Architecture – Hardware Switches
ToR ToR
Spine Spine
ToRVTEP VTEP
Cisco VTS
Hypervisor
VM
x86 Server
Hypervisor
VM
x86 Server
VTF VTF
Hypervisor
VMVM
x86 Server
REST API
DCI
NX-API, CLI, YANG
VTEP
Border Leaf VTEP
VMware vCenter
Cisco Confidential 31© 2013-2014 Cisco and/or its affiliates. All rights reserved.
VTS Architecture - VTF
Cisco VTS
ToR ToR
Spine Spine
ToR
Hypervisor
VM
x86 Server
Hypervisor
VMVM
x86 Server
REST API
DCI
NX-API, CLI, YANG
VTEP VTEP
Border Leaf VTEP
VMware vCenter Hypervisor
VM
x86 Server
VTF VTF
VTEP
User space, Multi-tenant, line rate packet forwarder
Uses Vector Packet Processing technology
Fully integrated with Intel DPDK
Supports VXLAN, can be extended to support MPLSoGRE, L2TPv3, MPLSoUDP, native MPLS and SR
Programmed by VTS using Restconf/YANG
Cisco Confidential 32© 2013-2014 Cisco and/or its affiliates. All rights reserved.
VTS Functionality
• Discover ToRs, Servers and interconnections• Manage switch and network topology status• Topology information via API or GUI
Discovery
• VXLAN Provisioning (BGP EVPN & Flood/Learn) • VXLAN Overlay management (Add/Modify/Delete)• Multi-tenancy support• Track and Update VNIDs as VM moves• Network facing resource management
Provisioning
• Tenant to VNID mappings and VNID status• VNID to VTEP mappings• VTEP to VLAN and end host mapping• Trace VMs connected to VTEP• VTEP status within a VNID• VXLAN and fabric statistics
Overlay Visibility
33© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Unified Open NX-OS
Cisco Confidential 34© 2013-2014 Cisco and/or its affiliates. All rights reserved.
ExtensibilityAuto DeploymentOptions
Open ApplicationIntegration
ProgrammabilityTool Choice
DevOpsEnabling
POAP NXAPI
Yocto SDK
Standard Open InterfacesOpen Interfaces Automation and
VisibilityAdaptable NXOSAdaptableSDK
Programmable BootStrap and Provisioning
Package and Application Management
Native Application Integration
PXEData
Models
Server Management Tools
OPEN NX-OS - Extensible, Open, Programmable
34
Cisco Confidential 35© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Open NX-OS: Infrastructure Layer Enhancements
35
OPEN BOOTLOADERS & PROVISIONING
OPEN PACKAGE/APPLICATION INTEGRATION
OPEN INTERFACES
OPEN OBJECT BASED API’s (NX-API, Model Driven)
Open NX-OS consistent across both ToR and Modular
Open NXOS
Cisco Confidential 36© 2013-2014 Cisco and/or its affiliates. All rights reserved.
VTEP VTEP VTEP VTEP
• Leverage existing compute deployment infrastructure (PXE/iPXE) for operationalizing NX-OS
• Deploy NX-OS from a web server via HTTPS or TFTP server with support for both IPv4 and IPv6
• NX-OS CLI option added to select boot option either <bootflash(default) > or <pxe>
Boot Server(DHCP & HTTP/TFTP)NX-OS Image Repository
DHCP DISCOVER(v4/v6)
IP Address & File/Image URL
TFTP GET FILE/HTTP URL
http://n9k-dk9….bin..
Validate Image Checksum & Boot
Open NX-OS Bootloaders & ProvisioningiPXE
36
Cisco Confidential 37© 2013-2014 Cisco and/or its affiliates. All rights reserved.
•Ability to third party packages in Secure Guestshell or natively in NX-OS kernel
• Install all third party applications (Puppet/Chef, etc) as RPMs
•Daemon managed via standard Linux interfaces•Built-in support for YUM package manager•Patching and upgrade using standard rpm/yum workflows
• NX-OS processes(BGP) can be upgraded/patched via “yum update”
37
Package as RPM
C app with standard Linux
constructs
Open Embedded 64
bit Build Environment
Cisco/Local
Repository
RPM local repository
RPM uploadYUM Install
Linux Daemon
Linux Kernel
• Raw Socket • Netdevs• Libpcap
init.d
Monitoring
server
ASIC
Build Server Target Switch
Open NX-OS Package Management via YUM/RPMLXC and Native Daemons
Cisco Confidential 38© 2013-2014 Cisco and/or its affiliates. All rights reserved.
•Leverage Linux command toolkit for monitoring configuration and troubleshooting
# tcpdump -w file.pcap -i eth1-1Use ethtool to display detailed interface statistics:
#ethtool –S eth2-1 Use ifconfig to change mtu for an interface to
jumbo MTU:#ifconfig eth2-1 mtu 9000
Use ip route to add a static route for a given interface:#ip route add 203.0.113.0/24 via 198.51.100.2
dev eth2-1Leverage bash for NX-OS scripting automation
vsh –c “show interface brief” | grep up | awk/sed
38
Open NX-OS Linux InterfacesBash Access
Cisco Confidential 39© 2013-2014 Cisco and/or its affiliates. All rights reserved. 39
• Tool provides a convenient way for network engineers to get up to speed with scripting and automation via web browser interface
• Available on all Nexus platforms.
• CLI commands embedded in structured input and output (JSON/XML) via HTTP/HTTPS
• Use “feature nxapi” to enable access on the platform
Open NX-OS ProgrammabilityNX-API Developer Sandbox
Cisco Confidential 40© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Updates on Nexus Portfolio Offerings
Programmable NetworkApplication Centric Infrastructure
NEW! Unified Open NX-OS Release for Nexus 3000 and Nexus 9000 (Q3 2015)• Enhancements to NX-API – object store
and model driven• Native 3rd party RPM applications
integration (tcollector, Nagios, Ganglia, Puppet / Chef etc.)
• Linux utilities support for seamless tool integration across compute and network
• SDK for custom application integration
NEW! ACI Release for Nexus 9000 (Shipping June 2015 )• Microsoft Azure and System Center
Integration• Programmability examples: vCenter plug-
in, ACI toolkit etc. • Simplified operations • Stretched fabric, multiple destinations
from 30KMs to 150KMs• Group-based policy on Openstack• New ACI ecosystem partners (CliQr)
DBDB
Web Web App Web App
NEW! Common NX-API across N2K-N9K (2H 2015)
Programmable Fabric
NEW! Virtual Topology System (VTS) for software overlay provisioning and management across for Nexus 2K-9K (2H 2015)• Standards-based fabric
support on Nexus 5600/7x00 with VXLAN BGP EVPN (shipping with Nexus 9000 today)
VTS
Cisco Confidential 41© 2013-2014 Cisco and/or its affiliates. All rights reserved.
Nexus 9000® SeriesYour Deployment, This Makes it Happen!
Cisco Nexus 9300Platform Fixed Switches
NX-OS and ACI
Choice of Fabric Architectures
Feature Consistency with Silicon Innovations
Cisco Nexus 9500Platform Modular Switches
Nexus 9516 – Best of Interop Data Center 2014 APIC – Best of Interop SDN 2015
In partnership with: