cisco nx-os software architectured2zmdbbm9feqrf.cloudfront.net/2014/usa/pdf/brkarc-3471.pdfcisco...
TRANSCRIPT
Cisco NX-OS Software Architecture
BRKARC-3471
Ron Fuller, CCIE#5851 (R&S/Storage)
Technical Marketing Engineer, Nexus 7000
@ccie5851
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Agenda
• NX-OS Origins & Overview
• NX-OS Modular Architecture
• High-Availability Infrastructure
• High-Availability Features & Capabilities
• Command Line Interface
• Operational & Management Features
• Licensing & Lifecycle
• Innovation
• Conclusion
3
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS: Designed for the Data Center
NX-OS SAN-OS
IOS
CatOS
MDS 9000
Catalyst 6500
4
Nexus 9x00/7x00/6000/5x00/
4000/3000/1000V
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Cisco NX-OS Adoption
5
Shipping for 6+ years
50,000+ customers
600,000 systems
Validated Design Guides and Case Studies
Differentiating features driving the adoption and being deployed
Major Certification and Deployment milestones
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Uptime
Feltham, England
Kernel uptime is 1313 day(s)
Nexus 7018
Lawrenceville, Georgia
Kernel uptime is 1383 day(s)
Nexus 5010
Kernel uptime is 1343 day(s)
MDS9513
Eschbon, Germany
Kernel uptime is 1336 day(s)
NEXUS 5020
Kernel uptime is 1567 day(s)
NEXUS 7018
NY, New York
Kernel uptime is 1333 day(s)
MDS IBM FC Bladecenter
Hong Kong
Kernel uptime is 1348 day(s)
Nexus 7010
UK
Kernel uptime is 1330 day(s)
Nexus 7018
Houston
Kernel uptime is 958 day(s)
MDS
Malaysia
Kernel uptime is 1182 day(s)
Nexus 7010
> 3.5Y RTP, North Carolina
Kernel uptime is 1363 day
Nexus 5010
> 3.5Y
> 3.5Y
> 1.5Y
> 2.5Y
Chicago
Kernel uptime is 675 days
MDS
Germany
System uptime: 1619 days
Nexus 7010
Rome, Italy
Kernel uptime is 1813 day(s),
Nexus 7010
Ireland, UKI
System uptime: 2612 days
MDS9509
> 4.5Y > 7.0Y
> 3.5Y
> 3.5Y
> 3.5Y
> 4.0 Y
> 4.0 Y
> 3.0 Y
> 3.5 Y
Italy
1364 days
MDS 9513
> 3.5Y
Sweden
1475 days
Nexus 7018
> 4.0 Y
VI, Virginia Beach
Kernel uptime is 1373 day(s)
Nexus 7010
6
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Nexus Validation Testing (NVT)
• NVT provides a comprehensive, end-to-end systems test of Nexus powered data centers
• Includes Nexus 7000, 6000, 5000, 3000, and UCS in multiple Pods and Data Centers
– IPv4 & IPv6
– FabricPath, vPC, vPC+ and OTV
• Testing on pre-Cisco.com images
7
http://www.cisco.com/c/en/us/td/docs/switches/datacenter/sw/nvt/index.html
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Operating System
Data Center Network Manager (DCNM)
Nexus 2000 Nexus 3000
Nexus 1000V Nexus 7x00 MDS 9x00 Nexus 5x00
Nexus 4000
The Cisco Unified Fabric Family
8
• Complete data center class switching portfolio
• Consistent operating system across all platforms
• Infrastructure scalability, transport flexibility and operational manageability
Nexus 6000 Nexus 9x00
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Comprehensive Data Center Feature Set
9
Engineered to meet evolving data center landscape
• IPv4, IPv6 • Multicast • Interface / L2 / L3 Scale • WCCP L2 • Security • DHCP v6 Relay • VRRPv3
• Python • onePK • OpenFlow • Chef and Puppet Agents • Containers
• OTV • LISP • VPLS • OTV + VLAN Translation
• DCB/FCoE • Unified Ports • Multi-Hop FCoE
• Virtual Port Channel (VPC)
• Fabric Extender (FEX)
• eVPC
• FabricPath
• FabricPath Multi-Topology
• Anycast HSRP
• DFA– Enhanced Forwarding
• VRF
• VDC (4 8)
• VM FEX / Adapter FEX
• QinQ
• MPLS L3 VPN
• VXLAN
• DFA - SegmentID
• SNMP / XML
• Netflow (Full and Sampled)
• IPSLA
• SPAN/ERSPAN +
enhancements
• Advanced Network Analytics
• Single Point of Management
• Hitless ISSU
• Non-Stop Forwarding
• Stateful Switchover
• BFD
• BFD additional clients
• BGP PIC Edge
• Software Patching
Roadmap generally applicable to Nexus 7K, 6K, 5K, 3K
R&S Baseline SDN / Programmability Data Center Interconnect Storage Convergence
Fabric Technologies Virtualization / Multi-
Tenancy Monitoring / Management High Availability
• Industry leading Data Center solutions
• Enable Catalyst to Nexus migration for DC use-cases
• Focus on Operational excellence NX-OS Focus
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Certifications for NX-OS
10
• IPv6 Ready Logo Phase I Certified https://www.ipv6ready.org/db/index.php/public/logo/01-000556/
• FIPS 140-2 Certified – Completed in April 2011 - Cert# 1533, 1534
– http://csrc.nist.gov/groups/STM/cmvp/documents/140-1/140val-all.htm
• EAL4 Common Criteria Certified – Completed in April 2011
– http://www.niap-ccevs.org/st/vid10349
• NX-OS 6.2 is targeted for IPv6 Phase II and FIPS 140-2 and CC certification
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Distributed Architecture Distributed Forwarding and Control-plane
11
• OS designed to leverage distributed hardware architecture
• Fabric & forwarding engine removed from supervisor
• Each I/O module has independent control-plane and forwarding hardware
• Control-plane & data-plane separation (same on Nexus 6000 & 5x00)
• Fully distributed system for non-disruptive SSO & ISSU (SSO only available on dual-sup Nexus 7x00 and 9500)
Supervisor
(Control-Plane)
Fabrics
I/O Module
(Forwarding Engine)
EO
BC
* Module 1
Module 2
Module 3
*EOBC: Ethernet Out Of Band Channel
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Agenda
• NX-OS Origins & Overview
• NX-OS Modular Architecture
• High-Availability Infrastructure
• High-Availability Features & Capabilities
• Command Line Interface
• Operational & Management Features
• Licensing & Lifecycle
• Innovation
• Conclusion
12
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Modular Architecture
13
Kernel
Netstack
Management
Infrastructure
Hardware
Drivers
Feature API
API
Feature Velocity
Faster Defect Resolution
Consistency
HA
Infrastructure
API
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Kernel
14
• Linux 2.6 kernel – N7K, N6K, N5K
• Linux 3.4 kernel – N9K
• Brings the benefits of Linux
oResilient Pre-emptive Multitasking Multi-threaded
oScalable Multi-CPU/Core support
oConstant development and enhancement
Kernel
Netstack
Management
Infrastructure
Hardware
Drivers
Feature API
API
HA
Infrastructure
API
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Platform Specific Portion
15
• Chipset specific code
• Provide Hardware Abstraction Layer (HAL)
• Ported per platform
Kernel
Netstack
Management
Infrastructure
Hardware
Drivers
Feature API
API
HA
Infrastructure
API
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Netstack
16
• Complete network stack implemented in user space • L2 Packet Management/ARP
• IPv4/IPv6
• ICMPv4/ICMPv6
• TCP/UDP & Socket Library
• Added Functionality • Virtualization (VDCs/VRFs)
• High-Availability (SSO)
• Added system stability Kernel
Netstack
Management
Infrastructure
Hardware
Drivers
Feature API
API
HA
Infrastructure
API
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Management Infrastructure
17
• Provides CLI and configuration interfaces
• Provides SNMP agent
• Provides NETCONF/XML interface
• Provides Python
• Provides Cisco ONE Kernel
Netstack
Management
Infrastructure
Hardware
Drivers
Feature API
API
HA
Infrastructure
API
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Python Software Architecture
18
CLI Interpreter
Python
Interpreter
Other NX-OS component
(BGP, OSPF, etc…)
Operator
Console/Telnet/SSH
Python can run from the switch CLI
Python script can be run once or at specific
intervals
Configuration / show commands can be
executed from Python Interpreter mode
Call a different python script from a script
Ability to automatically run at bootup
Parse show outputs and perform
conditional actions (syslog, email, traps)
Integration with PoAP
Check RIB/FIB tables
Restrict access (security)
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Feature/Service Granularity
19
• Highly granular implementations
• Each service is an individual memory protected process – Including multiple instances of
particular service
• Effective fault isolation between services
• Individually Monitored & Managed
Kernel
Netstack
Management
Infrastructure
Hardware
Drivers
HA
Infrastructure
UDLD FCF FCoE STP
HSRP OTV vPC LISP
OSPF EIGRP BGP PBR
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Conditional Features
20
• Services (Protocols/Features) can be explicitly enabled/disabled N7K-3-Core1(config)# feature ?
<snip>
lacp Enable/Disable LACP
ldap Enable/Disable ldap
lisp Enable/Disable Locator/ID Separation Protocol (LISP)
lldp Enable/Disable LLDP
msdp Enable/Disable Multicast Source Discovery Protocol (MSDP)
mvrp Enable/Disable MVRP
netflow Enable/Disable NetFlow
ntp Enable/Disable NTP
ospf Enable/Disable Open Shortest Path First Protocol (OSPF)
<snip>
• Disabling a service releases associated resources, configuration and CLI
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32
Forwarding Engine VOQs VOQs
Fabric ASIC
To Fabric Modules
10G MAC 10G MAC 10G MAC 10G MAC
4:1 Mux 4:1 Mux 4:1 Mux 4:1 Mux 4:1 Mux 4:1 Mux 4:1 Mux 4:1 Mux
10G MAC 10G MAC 10G MAC 10G MAC
Replication
Engine
Replication
Engine
Replication
Engine
Replication
Engine
Front Panel Ports
LC
CPU
EOBC To Central Arbiter
NX-OS Runs on the Linecard
21
• Microcode version of NX-OS powers the linecards
• Runs on linecard control-plane CPU
• Service processes on the linecards are for hardware and functional support
• Reinforces highly distributed architecture
• In Service Upgrade capabilities (ISSU)
Kernel
Netstack
Management
Infrastructure
Hardware Drivers
HA
Infrastructure
Port Manager
ACL/QoS
Manager
FIB Manager
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Control-Plane/Data-Plane Separation
22
I/O Module
Static RIP EIGRP IS-IS OSPF BGP
Unicast Routing Information Base (uRIB)
uFDM
uFDM & FIB Manager
FIB Hardware
Supervisor
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Platform Packaging and Delivery
23
• Modular nature of NX-OS allows delivery of “permutations” based on hardware capabilities
• Kernel, core infrastructure code, and APIs remain consistent
• Minimizes development
• Maximizes code reuse & feature velocity
Kernel
Netstack
Management
Infrastructure
Hardware Drivers
HA
Infrastructure
UDLD FCF FCOE STP
ACL &
QoS OTV vPC LISP
OSPF EIGRP BGP PBR
Kernel
Netstack
Management
Infrastructure
Hardware Drivers
HA
Infrastructure
UDLD FCF FCOE STP
ACL &
QoS OTV vPC LISP
OSPF EIGRP BGP PBR
Kernel
Netstack
Management
Infrastructure
Hardware Drivers
HA
Infrastructure
UDLD vPath FCOE LACP
ACL &
QoS OTV vPC LISP
OSPF EIGRP BGP PBR
Nexus 7x00
Nexus 6000/5x00 Nexus 1000v
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Agenda
• NX-OS Origins & Overview
• NX-OS Modular Architecture
• High-Availability Infrastructure
• High-Availability Features & Capabilities
• Command Line Interface
• Operational & Management Features
• Licensing & Lifecycle
• Innovation
• Conclusion
24
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS High-Availability Infrastructure
25
• Actually composed of 3 sub-services
–System Manager
–Message & Transaction Service (MTS)
–Persistent Storage Service (PSS)
Kernel
Netstack
Management
Infrastructure
Hardware
Drivers
Feature API
API
HA
Infrastructure
API System
Manager
PSS
MTS
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS System Manager
26
• Center of service management and fault recovery
• Acts like Unix-like ‘init’ process.
• Starts up configured features/services
• Heartbeats received from services
HA
Infrastructure
System
Manager
PSS
MTS
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Message & Transaction Service
27
• Message relay system for IPC communications
• Provides reliable unicast & multicast delivery
• Used for service-to-service and module-to-module messaging
HA
Infrastructure
System
Manager
PSS
MTS
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Persistent Storage Service
28
• Lightweight key/value database
• Provides store options for DRAM or NVRAM
• API for services to store data
• Used to maintain runtime data/state
• PSS updated in NX-OS 6.2 to increase overall scale
• PSS in NX-OS 6.2 is multi-core and multi-CPU “aware”
HA
Infrastructure
System
Manager
PSS
MTS
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Agenda
• NX-OS Origins & Overview
• NX-OS Modular Architecture
• High-Availability Infrastructure
• High-Availability Features & Capabilities
• Command Line Interface
• Operational & Management Features
• Licensing & Lifecycle
• Innovation
• Conclusion
29
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Stateful Process Restart
30
If a fault occurs in a process…
• HA manager determines best recovery action (restart process, switchover to redundant supervisor)
• Process restarts with no impact on data plane
• State is recovered, operation resumes
• Total recovery time: ~10s ms
Restart
process!
Kernel
Netstack
Management
Infrastructure
Hardware
Drivers
HA Infrastructure
UDLD SSH IGMP STP
HSRP 1 OTV vPC HSRP 2
OSPF 1 EIGRP BGP OSPF 2
Control-Plane
Data-Plane
NX-OS services checkpoint their runtime state to the PSS for recovery in the event of a failure
OSPF
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Active and Standby Supervisor Syncing
31
Services start in standby mode
Active SUP
PSS
Service
System manager
MTS
Standby SUP
PSS
System manager
MTS Standby Online (all services gsync)
Service
Determine
Active/Standby
1
Request Initial States (gsync) 2
Snapshot of Initial
States
3
Services Set Initial
States
4
Event-driven
Syncing
5
Initial State for
Services Runtime config
Runtime states/data
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Stateful Supervisor Switchover
32
• Active/Standby
• Fast switchover time – State is already in place
• Switchover initiated if:
repeated critical process restart failures
kernel failures
supervisor hardware failure detected by diagnostics (GOLD)
Nexus-Dual-Sup# show system redundancy status Redundancy mode --------------- administrative: HA operational: HA This supervisor (sup-1) ----------------------- Redundancy state: Active Supervisor state: Active Internal state: Active with HA standby Other supervisor (sup-2) ------------------------ Redundancy state: Standby Supervisor state: HA standby Internal state: HA standby
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
In-Service Software Upgrade
33
N7K# install all kickstart bootdisk:6.2-kickstart system bootdisk:6.2-system N7K#
N7K#
Sup 2 Sup 1
Upgrade standby supervisor 1
Reload standby supervisor 2
Upgrade standby supervisor 4
Perform SSO 3
Upgrade LCs & FEX in series* 6
Release
6.2 Reload standby supervisor 5
Release 6.1 • Parallel upgrade of the I/O modules supported on the
Nexus 7000 from 5.2 (3 at the same time)
• Parallel upgrade of FEX supported on the Nexus 7000 from
6.1 (10 at the same time)
Active
5.2 Active
6.2
Standby
6.2 Standby
5.2
Standby
6.2 Standby
5.2
Release
6.2
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Hitless ISSU on the Nexus 5x00/6000
34
• Difference in the detailed operation from Nexus 7K
– Single supervisor/control-plane vs. dual supervisor
– L3 ISSU not supported on the 5K/6K
– ISSU not possible if non-edge STP designated port (only works in the access)
• During ISSU, control plane functions are temporarily suspended.
• Control Plane restored within 80 seconds
• Hitless ISSU of the Nexus 2000s (Nexus 5x00/6000)
• Supported from NX-OS 4.2(1)N1
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Agenda
• NX-OS Origins & Overview
• NX-OS Modular Architecture
• High-Availability Infrastructure
• High-Availability Features & Capabilities
• Command Line Interface
• Operational & Management Features
• Licensing & Lifecycle
• Innovation
• Conclusion
35
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS CLI Highlights Improved Over IOS Model
36
N7K(config)# int e1/1 N7K(config-if)# ip address 192.168.0.1/23
Support for CIDR ‘slash’ notation for IPv4/IPv6 masks
N7K(config)# show interface e1/1 Ethernet1/1 is up Hardware: 10/100/1000 Ethernet, address: 001b.54c1.5d44 (bia 001b.54c1.5d44) MTU 1500 bytes, BW 100000 Kbit, DLY 10 usec, reliability 255/255, txload 1/255, rxload 1/255 <snip>
Hierarchy Independent CLI allows ‘show’ commands to be executed from exec-mode or config-mode
N7K# show cli history ? <CR> config-mode Display history of config commands only exec-mode Display history of exec commands only this-mode-only Display history from current mode only unformatted Display just the commands N7K# show cli history config-mode 12 05:20:34 int e1/1 13 05:20:42 where detail
Mode-aware CLI history
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Review Configuration with Flexibility
37
N7K# show running-config ntp ntp server 171.68.10.80 use-vrf management ntp server 171.68.10.150 use-vrf management ntp source 172.26.244.101 clock format 12-hours clock format show-timezone
Compare between startup- and running-configuration N7K# copy running-config startup-config [########################################] 100% N7K# config terminal Enter configuration commands, one per line. End with CNTL/Z. N7K(config)# feature telnet N7K(config)# sh running-config diff *** Startup-config --- Running-config *** 1,11 **** feature lacp --- 1,11 ---- + feature telnet feature lacp
Identify the line number and
difference between startup-
config and running-config
Display feature-specific configuration
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Running-config Permutations
38
• “show running-config” provides many enhancements
N7K# show running-config ?
<CR>
> Redirect it to a file
aaa Display aaa configuration
all Current operating configuration with defaults
am Display am information
arp Display arp information
bgp Display bgp information
callhome Display callhome configuration
cdp Display cdp configuration
cmp Display CMP information
copp show running config for copp
dhcp Display dhcp snoop configurations
diagnostic Display diagnostic information
diff Show the difference between running and startup configuration
dot1x Display dot1x configuration
eem Show the event manager running configuration
eigrp Display eigrp information
icmpv6 Display icmpv6 information
igmp Display igmp information
interface Interface configuration
ip Display ip information
ipqos show running config for ipqosmgr
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS CLI Tips
39
Dude, where am I?!
*Who* typed that command?!
N7K-3-Core1(config-router)# where
conf; router ospf 100 admin@N7K-3-Core1%default
N7K-3-Core1# show accounting log
Tue Apr 22 08:04:38 2014:type=update:id=vsh.19970:user=chad:cmd=switchto ; confi
gure terminal ; interface Vlan616 ; ip access-group mgt-outbound out (SUCCESS)
Tue Apr 22 08:04:38 2014:type=update:id=vsh.19970:user=chad:cmd=switchto ; confi
gure terminal ; interface Vlan616 ; no ip redirects (SUCCESS)
I don’t want to read the entire log N7K-3-Core1# show log last 10
2014 May 14 04:59:53 N7K-3-Core1 %BFD-5-SESSION_MOVED: BFD session 0x4200000d: I
nstalled on LC 5
2014 May 14 04:59:54 N7K-3-Core1 %BFD-5-SESSION_CREATED: BFD session to neighbor
10.1.0.9 on interface Eth5/1/1 has been created
Typing is hard N7K-3# show cli alias
CLI alias commands
==================
agg :switchto vdc Agg1
agg2 :switchto vdc Agg2
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
CLI Parsing Many options to parse output N7K-3-Core1# show int e5/1/1 | ?
cut Print selected parts of lines.
diff Show difference between current and…
temp files: remove them with 'diff…
on commands with big outputs, like…
egrep Egrep - print lines matching a pattern
grep Grep - print lines matching a pattern
head Display first lines
human Output in human format
last Display last lines
less Filter for paging
no-more Turn-off pagination for command output
section Show lines that include the pattern…
that are more indented than matching…
sed Stream Editor
sort Stream Sorter
source Run a script (python, tcl,...) from
bootflash:scripts
top Run the command before the pipe in a loop with
set…
tr Translate, squeeze, and/or delete characters
uniq Discard all but one of successive identical lines
vsh The shell that understands cli command
wc Count words, lines, characters
xml Output in xml format (according to .xsd defini…
xmlin Convert CLI show commands to their XML formats
xmlout Output in xml format (according to the latest …
xpath Run xpath query on xml output (to be used after…
begin Begin with the line that matches
count Count number of lines
end End with the line that matches
exclude Exclude lines that match
include Include lines that match
40
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
CLI Parsing - continued
41
N7K-3(config)# cli alias name cop sh pol int con | eg
"class|violate(d| rate)" | sed "s/\(.*class-map.*\) (match-
any)/\n\1/" | eg -v "violate rate 0 bytes/sec“
N7K-3# cop
class-map copp-system-p-class-critical
violated 0 bytes,
violated 0 bytes,
class-map copp-system-p-class-important
violated 0 bytes,
violated 0 bytes,
<snip>
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
IOS to NX-OS Conversion Tool
42
• Available today on cisco.com http://tools.cisco.com/nxmt
• Migrate Catalyst 6500/4500 configuration to Nexus 7x00/5x00/6000
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Agenda
• NX-OS Origins & Overview
• NX-OS Modular Architecture
• High-Availability Infrastructure
• High-Availability Features & Capabilities
• Command Line Interface
• Operational & Management Features
• Licensing & Lifecycle
• Innovation
• Conclusion
43
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Embedded WireShark Analyzer (Ethanalyzer)
44
• Real-time, on-the-device protocol analyzer provide ultimate visibility into various traffic hitting CPU from remote locations
Control
Processor Data
Traffic mgmt0
Inband
Monitor traffic from inband and
mgmt0 interfaces to the Control
Processor
Extensive capture and display
options, including to file (.pcap)
Capture rules/filters
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Configuration
45
• Facilitate change-management with configuration snapshots
• Checkpoint & Rollback
System and user generated checkpoints
System checkpoint automatically created
when any conditional features are
disabled
User-defined checkpoint can be initiated
from CLI
Rollback to any checkpoint allows easy
recovery
Current Running
Configuration
System
Checkpoint
New Running
Configuration
User-Defined
Checkpoint
Ro
llb
ack
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Checkpoint & Configuration Rollback
46
N7K(config)# no feature vpc N7K(config)# sh checkpoint summary System Checkpoint Summary ------------------------------------- 1) system-fm-vpc: Created by admin Created at Fri, 16:51:40 06 Nov 2009 Size is 24,567 bytes Description: None
System-checkpoint created automatically upon feature removal
User-defined checkpoint with description simplifies configuration management N7K# checkpoint 2009-11-06 description SQL DC ACL Update N7K# sh checkpoint summary User Checkpoint Summary ------------------------------------- 1) 2009-11-06: Created by admin Created at Fri, 18:33:41 06 Nov 2009 Size is 25,773 bytes Description: SQL DC ACL Update
Flexible option for configuration rollback N7K# rollback running-config checkpoint 2009-11-11 ? <CR> atomic Stop rollback and revert to original configuration (default) best-effort Skip errors and proceed with rollback stop-at-first-failure Stop rollback at the first error verbose Show the execution log
Default name for system-checkpoint,
‘system-fm-xxx’
Timestamp of checkpoint
help configuration
management
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Control Plane Policing (CoPP)
47
• Protects switch’s CPU from network traffic and improves stability of the platform
• Only the traffic sent to the CPU via the inband interface is subject to CoPP
– ARP, ICMP, SNMP, routing protocols, etc.
• Default CoPP policy works in majority of environments
– Can be customized for specific requirements
• Application requirements and/or scale
– Life cycle management of CoPP critical
• Monitor, tune, monitor, evaluate, repeat
• CoPP is updated to include new features & protocols
Linecard
FE
Linecard
FE
Transit
Packets
Transit
Packets
Layer 2 Protocols Layer 3 Protocols
VLAN PVLAN
OSPF BGP
EIGRP
GLBP HSRP IGMP
UDLD CDP
802.1X STP LACP PIM CTS SNMP
… …
Control Plane
Supervisor
Logical Representation of the Fabric Modules
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Embedded Event Manager (EEM)
• Action can be
• CLI Command
• Python Script
• Multiple actions per Event
• Rich set of Events
• Syslog messages
• Monitoring for certain CLI commands
• Memory thresholds
• Module status changes
• Missing fan tray
• Temperature thresholds event manager applet track_1_18_down
event track 1 state down action 1 syslog msg EEM applet track_1_18_down shutting down port eth1/33 as 1/18 went down action 2 cli conf term action 3 cli interface ethernet 1/33 action 4 cli shut
event manager applet track_1_18_up
event track 1 state up action 1 syslog msg EEM applet track_1_18_up bringing up port eth1/33 as 1/18 came up action 2 cli conf term action 3 cli interface ethernet 1/33 action 4 cli no shut
EEM to track interface down
EEM to track interface up
48
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Scheduler • Multiple schedules and multiple jobs per schedule
• Frequency
• Run Once, Daily, Weekly, Monthly
• Delta (Begin Job at specified time and repeat at specified intervals)
• Run CLI commands or Python scripts
Schedule 1
Run Weekly
Schedule 2
Run Once
Job 2 – Run
Sanity Check
Python Script
Job 1 – Full
System Check
…
Schedule 3
……..
Job 1 – Backup
Running
Configuration Job …… Job …… Job ……
49
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Scheduler Setup
# configure terminal # feature scheduler
# configure terminal # scheduler logfile size 32
# configure terminal # scheduler job name bkpConfig # cli var name timestamp $(TIMESTAMP) # copy running-config bootflash:/$(SWITCHNAM)-cfg.$(timestamp)
# configure terminal # scheduler schedule name bkpConfig # job name bkpConfig # time daily 23:00
Enable schedule
Configure log file size (kb)
Create job
Schedule job
50
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Stateful Process Restart & Patching
51
When a process is patched…
• Install process applies new patch
• HA manager restarts process
• Process restarts with patched code and no impact on data plane
• State is recovered, operation resumes
• Total recovery time: ~10s ms
Restart
process!
Kernel
Netstack
Management
Infrastructure
Hardware
Drivers
HA Infrastructure
UDLD SSH IGMP STP
HSRP 1 OTV vPC HSRP 2
OSPF 1 EIGRP BGP BGP
Control-Plane
Data-Plane
• NX-OS services checkpoint their runtime state to the PSS for recovery in the event of a failure
• Install process can use PSS state to recover state BGP
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
SMU Repository
Copy to Device
SMU Removed
Software Maintenance Update (SMU) Workflow (CLI)
Show Install Active
Show Install Committed
Show Install Inactive
Show Install Packages
SMU Committed
SMU Committed
Router> Install Add
Router> Install Activate
Router> Install Commit Router> Install Deactivate
Router> Install Commit
Router> Install Remove
SMU
.
.
Memory: Process:
Memory: Process:
SMU Applied Memory: Process:
Memory: Process:
Memory: Process:
SMU
52
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Cisco Software Manager (Formerly SMU Manager)
Challenge # 1 – Too many devices to manage
Challenge # 2 - Find the appropriate SMU considering the OS Image, the Process, and the SMU interdependencies.
• Maintain inventory of Devices including Image and SMUs supported
• Maintain an inventory of SMUs available on CCO
• Recommend a SMU upgrade path per supported Device
• Support existing Data Center Orchestration Tools (i.e. Chef)
53
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
CSM
Patch Server
S/W Download via SCP etc.
Initiate S/W Install
Which SMUs are available per image ?
Software SMU:
• BugID
• Image Version
• Severity
• CCO SMU Profile
• Device Image/SMU Profile
• Create CHEF Databag
Internet
Facilitate SMU downloads
Download Databag and Databag items
Download Cookbook to all devices
Pull Databag Item & SMU
Pull Databag Item & SMU
Pull Databag Item & SMU
Pull Databag Item & SMU
SMU
SMU SMU
www.cisco.com
SMU on CCO SMU
Cisco Software Manager ( Formerly SMU Manager)
54
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Nexus Options
Custom
Listeners
Nexus DC Switch
NXOS
Linux Container hosting One
PK Apps
Thrift API
OMI
Server OpenFlow
Agent
CLI
Python SNMP Custom
Apps
OMI
Python
Apps
Traditional
Management
OpenFlow
Controller
55
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
OnePK Architecture – High Level View
Nexus DC Switch
NXOS
Linux Container hosting One PK Apps
Thrift API
OMI
Server
OpenFlow
Agent Chef Agent
Puppet
Agent Custom….
OpenFlow
Controller Chef Puppet OMI
Custom
App
56
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Agenda
• NX-OS Origins & Overview
• NX-OS Modular Architecture
• High-Availability Infrastructure
• High-Availability Features & Capabilities
• Command Line Interface
• Operational & Management Features
• Licensing & Lifecycle
• Innovation
• Conclusion
57
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Licensing
58
Grace Period
• Allows trying functionality without a licence for 120 days
• Periodic syslog, callhome and SNMP traps warning when grace period nears expiry
• Self generated license for 90 days (beyond grace period)
• https://tools.cisco.com/SWIFT/LicensingUI/loadDemoLicensee?FormId=4056
Time-Bound Licenses
• License with expiry date
• Used for demo or as an emergency
• Periodic syslog, callhome and SNMP traps warning when time bound license nears expiry
• After expiry date feature will continue to run if grace period has not been exhausted
License PAK
(product activation key)
www.cisco.com
PAK +
chassis serial #
<xml...
licA ...>
license
file
Licenses are enforced on the switch # show license host-id
License tied chassis serial # stored in dual redundant NVRAM
modules on backplane
Licenses are issued in the form of a digitally signed text file
# install license bootflash:N7K-1234.lic
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Platform Universal System Image License-based Feature Management
59
Single NX-OS System Image
A+B
A+B
+C
A+C
A+B
+D
A+C
+D
A+D
Multiple SW Images NX-OS Base (A)
Enterprise
License
(B)
Advanced
License
(C)
Storage
Features
(D)
Future
License
Packages
Finding the right image
can be a challenge
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Enterprise LAN
IP routing
OSPFv2
OSPFv3
IS-IS
BGP for IPv4 & IPv6
EIGRP for IPv4 & IPv6
BFD
IP Multicast
PIM: Sparse, Bidir, ASM and SSM for IPv4 & IPv6
Multicast Source Discovery Protocol (MSDP) for IPv4
PBR for IPv4 and IPv6
GRE Tunnels
TrustSec (SGTs and MACSEC)
Advanced
LAN
VDCs
vPC Port Profile WCCP Port Security GOLD EEM TACACS LACP ACL QoS STP
STP Guards UDLD CDP CoPP uRPF IP Source Guard DHCP Snooping CMP ISSU SSO
Dynamic ARP Inspection Smart Call Home SNMP 802.1x SPAN Netflow v5 and v9 IEEE1588 Base
Scalable Services
Enhanced L2
FabricPath
PONG
Intelligent Traffic Director (ITD)
MPLS
MPLS VPN
LDP
MPLS QoS
TE/FRR
mVPN
MPLS OAM
6PE/6VPE
NX-OS Software Packaging – Nexus 7x00 Nexus 7000 Overview
60
FCoE*
Multi-Hop FCoE
FCF
FIP
Storage
Enterprise
Inter VSAN routing
VSAN based access control
* Per Module-based license
Transport
Services
OTV
LISP
+4 VDCs
VDC
Unlocks HW resources on XL Module
Scalability
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Base Enterprise
• SVI routed interfaces
• L3 routed ports on non-FEX interfaces
• Static Routing
• RIPv2
• EIGRP for Routed Access (Stub)
• OSPFv2 and OSPFv3
• HSRP
• VRRP
• IGMPv1, v2, v3
• PIM v2 (sparse)
• Routed ACLs
• uRPF
• MSDP
Enterprise LAN
BGPv4
EIGRP
VRF-Lite
v6 Routing (IS-ISv6, BGPv6)
HSRPv6/VRRPv3
ISSU vPC Port Profile LACP ACL QoS STP LLDP XML SNMP 1588 Base
Enhanced L2
FabricPath
FCoE License
FCoE:
• Native Fibre Channel
• FCoE
• NPV
• FC Port Security
• Fabric Binding
NX-OS Software Packaging – Nexus 5x00 Nexus 6000 Overview
61
VM-FEX
VM-FEX
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Base Enterprise
• SVI routed interfaces
• L3 routed ports on non-FEX interfaces
• Static Routing
• RIPv2
• EIGRP for Routed Access (Stub)
• OSPFv2 and OSPFv3
• HSRP
• VRRP
• IGMPv1, v2, v3
• PIM v2 (sparse)
• Routed ACLs
• uRPF
Enterprise LAN
BGPv4
EIGRP
VRF-Lite
v6 Routing (IS-ISv6, BGPv6)
HSRPv6/VRRPv3
ISSU vPC Port Profile LACP ACL QoS STP LLDP XML SNMP 1588 Base
Enhanced L2
FabricPath
FCoE License
Storage Services:
• 8-port FC/FCoE License
• Chassis license for 5548 and 5596
• FcoE NPV (available also as standalone license)
NX-OS Software Packaging – Nexus 600x Nexus 5500 Overview
62
FCoE NPV License
FCoE NPV:
• FCoE NPV
VM-FEX
VM-FEX for 5548 and 5596 only
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Releases
63
• NX-OS on Nexus 7000 Minimum software Recommendation http://www.cisco.com/en/US/docs/switches/datacenter/sw/nx-os/recommended_releases/recommended_nx-os_releases.html
• NX-OS on Nexus 5000 Minimum software Recommendation http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/release/recommended_releases/recommended_nx-os_releases.html
• NX-OS Software Release Strategy Document http://www.cisco.com/en/US/prod/collateral/iosswrel/ps9494/ps9372/guide_c07-658595.html
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
5.2 is a long-lived NX-OS train on N7K and N5K (EOS)
6.2 is a long-lived NX-OS train on N7K and N5K/N6K
NX-OS Software Life Cycle
64
Short Lived Release
FCS EoSM EOL
Maintenance Release
(1 or 2 releases)
(8-12 weeks)
End of
Maintenance
EoS
12
Months 6
Months
End of
Support
54
Months
Long Lived Release
FCS EoSM EOL
Maintenance Release
Introduction Phase
(8-12 weeks)
Maintenance Release
Mature Phase
(6-12 months)
End of
Maintenance
EoS
Up to 36 Months 12 Months
End of
Support
48 Months
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Agenda
• NX-OS Origins & Overview
• NX-OS Modular Architecture
• High-Availability Infrastructure
• High-Availability Features & Capabilities
• Command Line Interface
• Operational & Management Features
• Licensing & Lifecycle
• Innovation
• Conclusion
65
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Virtual Device Contexts (VDCs)
66
• VDCs are not… – The ability to run different OS levels on the same box at
the same time
– based on a hypervisor model; there is a single ‘infrastructure’ layer that handles h/w programming…
• VDC—Virtual Device Context – Flexible separation/distribution of Software
Components
– Flexible separation/distribution of Hardware Resources
– Securely delineated Administrative Contexts
Infrastructure
Layer-2 Protocols Layer-3 Protocols
VLAN mgr
STP
OSPF
BGP
EIGRP
GLBP
HSRP
VRRP
UDLD
CDP
802.1X IGMP sn.
LACP PIM CTS SNMP
RIB RIB
Protocol Stack (IPv4 / IPv6 / L2)
Layer-2 Protocols Layer-3 Protocols
VLAN mgr
STP
OSPF
BGP
EIGRP
GLBP
HSRP
VRRP
UDLD
CDP
802.1X IGMP sn.
LACP PIM CTS SNMP
RIB RIB
Protocol Stack (IPv4 / IPv6 / L2)
Kernel
VDC A
VDC B
VDC A VDC B
VDC n
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Virtual Device Contexts (VDCs)
67
• Typical silo/stovepipe design • Production, Development, Test
• Intranet, Internet, DMZ, Extranet
• Application A, Application B, Application C
• Customer A, Customer B, Customer C
• VDCs enable collapsing of physical
infrastructure into logical infrastructure
• Preserves security, administration, and organizational
boundaries, & fault isolation
• FIPS 140-2 and Common Criteria EAL4+ certified
Physical network
islands are virtualized
onto common data
center infrastructure
VDC Extranet
VDC Prod
VDC DMZ
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Fabric Extenders
68
• Nexus 7x00/600x/5x00 + FEX is like a “Virtual Chassis” • Nexus 2000 FEX is an “intelligent patch panel” to its “parents” • No Spanning Tree between the FEX and its “parent” • No local switching on the FEX • NX-OS Linecard code runs on the 2148/2248/2232/2248
Fabric Extender (FEX)
Nexus 5x00 Nexus 7x00 Nexus 600x Nexus 9x00
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Access Layer with Nexus 2000
69
Physical view (Efficient cabling)
Logical view (Efficient Management)
Combines benefit of ToR and EoR architecture
• Reduces cable runs
• Reduce management points in the network
• Easier to ensure feature consistency across hundreds or thousands of server ports
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Virtual Port Channel (vPC) Objectives
70
• Provides a loop-free topology
• Maximises bandwidth / lower over-subscription
• Improved convergence & availability
vPC on Nexus
logical equivalent
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Up-to 32 Ports
vPC Topology Example
71
• Two layers of vPC peers can be connected back-to-back e.g. N7k to N5k
• Opportunity for very high bandwidth using an evolutionary development of STP
• Up to 32-way port-channel
*Note* Use unique domain IDs
• Back to Back
vPC member
Routed
Interface
Host Port
Nexus
7x00
Nexus
5x00/600x
Nexus
2000
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
FabricPath: an Ethernet Fabric Shipping on Nexus 7x00, Nexus 600x and Nexus 5x00
72
N7K(config)# interface ethernet 1/1
N7K(config-if)# switchport mode fabricpath
Eliminates Spanning tree limitations
High resiliency, fast network re-convergence
Any VLAN, Anywhere in the Fabric
Connect a group of switches using an arbitrary topology
With a simple CLI, aggregate them into a Fabric
FabricPath
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
FabricPath Emulated Switch ID
• Anycast HSRP data plane allows for binding an “Anycast Switch ID” (ASID) with the vMAC of HSRP group ID.
• ASID – uses a similar concept as an “Emulated Switch ID” deployed in vPC+ environments where every Anycast HSRP Gateway router apart from its real Switch-ID, also advertises ASID
• Since ASID is mapped to multiple switches, any InterVLAN or Routed traffic can leverage more than one exit point in the network*.
A B
VLAN 10 VLAN 20
WAN
L2
L3
C
HSRP vMAC
SID:10 SID:20 SID:30 SID:40
SID:50 SID:60 SID:70 SID:80
ASID ASID ASID ASID
*FabricPath IS-IS facilitates building ECMP with all available shortest paths
73
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Overload Bit
• Additional Spine switch is brought up and starts sending updates with Overload Bit set
s1 s2 s3 s4
FabricPath
Up
date
Advertise
overload bit
in topology
updates
Too many SPF updates: back off.
Routing table incomplete maintain overload bit, no blackholing
74
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Overload Bit - continued
• Spine clears “Overload bit” and now is ready to become a transit path for traffic
s1 s2 s3 s4
FabricPath
Up
date
Overload bit
cleared: S1 is
operational
Routing table complete, clear overload bit and start routing…
75
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Dynamic Fabric Automation (DFA)
76
Fabric
Management
Optimized
Network Virtual Fabrics Workload
Automation
Fabric
Management
Workload
Automation
Virtual Fabrics Optimized
Network
Bundled functions are modular and simplified for scale and automation
More details in BRKDCT-2385 – Dynamic Fabric Automation Architecture
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Today’s DC Challenges
Many devices to manually configure
Protocol Restrictive Function
Deficient SW overlays
Network elasticity constraints
Disparate workload provisioning
Static resource allocation
Auto-configuration at scale
Protocol independent Function Integration
HW-Based Fabric Optimized Functions
Any workload (physical/virtual) anywhere anytime
Automated Workload Workflow
Dynamic Resource Management
Operational Complexity
Architecture Rigidity
Infrastructure
Inefficiency
Are the result of…
SIMPLIFY
OPTIMIZE
AUTOMATE
Dynamic
Fabric
Automation
Architecture
77
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public 78
Advantages
Any subnet, anywhere, rapidly
Reduced Failure Domains
Extensible Scale & Resiliency
Profile Controlled Configuration
Any/all subnets on any leaf
Any/all Leaf Distributed Default Gateways
Full bisectional bandwidth (N spines)
Network Config profile Network Services Profile
n1000v# show port-profile name WebProfile
port-profile WebServer-PP
description:
status: enabled
system vlans:
port-group: WebServers
config attributes:
switchport mode access
switchport access vlan 110
no shutdown
security-profile Protected-Web-Srv
evaluated config attributes:
switchport mode access
switchport access vlan 110
no shutdown
assigned interfaces:
Veth10
Cisco Dynamic Fabric Automation Scale, Resiliency and Efficiency
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Any subnet anywhere => Any leaf can instantiate any subnet
All leafs share gateway IP and MAC for a subnet (No HSRP)
ARPs are terminated on leafs, No Flooding beyond leaf
Facilitates VM Mobility, workload distribution, arbitrary clustering
Seamless L2 or L3 communication between physical hosts and virtual machines
GW IP: 11.11.11.1
GW MAC:
0011:2222:3333
GW IP: 10.10.10.1
GW MAC:
0011:2222:3333
L3
L2
Anycast Gateway
DFA - Distributed Gateway at the Leaf
79
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Virtual Fabrics
Introducing Segment-ID Support
Traditionally VLAN space is expressed
over 12 bits (802.1Q tag)
Limits the maximum number of segments in a
data center to 4096 VLANs
DFA leverages a double 802.1Q tag for a
total address space of 24 bits
Support of ~16M L2 segment (10K targeted at
FCS)
Segment-ID is hardware-based innovation
offered by leaf and spine nodes part of the
Integrated Fabric
FabricPath Frame Format
Integrated Fabric Frame Format
Segment-ID =
80
802.1Q 802.1Q
DFA Frame
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Virtual Fabrics
802.1Q Tagged Traffic to Segment-ID Mapping
Segment-IDs are utilized for providing
isolation at L2 and L3 across the
Integrated Fabric
802.1Q tagged frames received at the
leaf nodes from edge devices must be
mapped to specific Segments
The VLAN-Segment mapping can be
performed on a leaf device level
VLANs become locally significant on the
leaf node and 1:1 mapped to a Segment-ID
Segment-IDs are globally significant,
VLAN IDs are locally significant
81
WAN
802.1q Trunk 802.1q Trunk
VLANs VLANs
Segment-IDs (Global)
Segment-ID
5000
vlan 10
mode fabricpath
vn-segment 5000
vlan 10
mode fabricpath
vn-segment 5000
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Simplifying Fabric Management & Optimizing Fabric Visibility
82
Advantages
Device Auto-Configuration
Cabling Plan Consistency Check
Automated Network Provisioning
Common point of fabric access
Network, vFabric & Host Visibility
TFTP
Services
DHCP
Services
XMPP
Server
LDAP
Message
Broker DCNM (CPoM)
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Workload Automation & Open Environment
83
Advantages
Any workload, anywhere, anytime
Open Integration: orchestration
Automated scalable provisioning
Workload aware fabric
Services
Controller
Fabric Mgmt
Provisioning
Open
APIs
Published
Schemas
Network & Network
Services Policies
Cloud Stacks
Compute & Storage
Policies
UCS Director
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Overlay Transport Virtualization Enable L2 Extension over IP Networks
84
West
OTV
East
South
MAC Addresses
Advertisements OTV
OTV
Custom built technology to solve specific challenges
• No pseudo-wire state maintenance
• Optimal multicast replication
• Multi-point connectivity
Active control plane protocol brings massive benefits
• Failure boundary preservation
• Built-in loop prevention
• Automated multihoming
• Site independence
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Cisco Integrated Traffic Director (ITD)
Benefits: • Order of magnitude OPEX savings : reduction in
configuration, and ease of deployment
• Order of magnitude CAPEX savings : Wiring,
Power, Rackspace and Cost savings
• Scalability : Multi-Terabits/s, large number of
nodes, no CPU overhead.
• High availability : N + M redundancy
ITD Overview: • ASIC based multi-Tbps L4 load-balancing at line-rate
• VIP based server load-balancing
• Capability to create clusters of devices, eg, Firewalls
• Redirect line-rate traffic to any devices, for example
web cache engines, WAE, video-caches, etc..
• No service module or external load-balancer needed.
• IP-stickiness
• ACL along with redirection and load balancing
simultaneously.
• Monitoring the health of servers/appliances.
• Supports both IPv4 and IPv6
ITD Deployment
Scalable solution for L3/L4 load-balancing, redirection and clustering
ITD
Redirect
Clients
ACL to select traffic
Select the traffic
destined to VIP
Load-balance
85 85
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Cisco Integrated Traffic Director (ITD) Enabling Scalable and highly available data-centers
Application/Services scaling
Multi-Tbps Scale
VIP based L3/L4 server load-balancing Redirect traffic to web cache,
video-cache, WAE, etc.
Create multi-Tbps firewall
Significant OPEX reduction
Investment protection : Supported on all LCs and Sups on both N7000 and N7700* *F1 Modules do not support ITD
86 86
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public 87
Problems being addressed:
VLAN scale – VXLAN extends the L2 segment ID field to 24-bits, potentially allowing for up to 16 million unique L2 segments over the same network
Layer 2 segment elasticity over Layer 3 boundary – VXLAN encapsulates L2 frame in IP-UDP header
High Level Technology Overview:
MAC-in-UDP encap
Leverages multicast in the transport network to simulate flooding behavior for broadcast, unknown unicast and multicast in the same segment
Leverage ECMP to achieve optimal path usage over the transport network
NX-OS and VXLAN What does VXLAN solve/address?
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS and VXLAN Supported Functionalities
88
VXLAN to VLAN Bridging (L2 Gateway) VXLANORANGE
Ingress VXLAN packet on
Orange segment
Egress interface chosen
(bridge may .1Q tag the packet)
VXLAN L2
Gateway
Egress is a tagged interface.
Packet is routed to the new VLAN VXLAN to VLAN Routing (L3 Gateway)
VXLANORANGE
Ingress VXLAN packet on
Orange segment
VXLAN
Router
Destination is in another segment.
Packet is routed to the new segment
VXLANORANGE VXLANBLUE
Ingress VXLAN packet on
Orange segment
VXLAN
Router
VXLAN to VXLAN Routing (L3 Gateway)
*Check hardware platform for capability
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Agenda
• NX-OS Origins & Overview
• NX-OS Modular Architecture
• High-Availability Infrastructure
• High-Availability Features & Capabilities
• Command Line Interface
• Operational & Management Features
• Licensing & Lifecycle
• Innovation
• Conclusion
89
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Software Architecture Top Things to Remember
90
• NX-OS built around High-Availability as a core principle
• NX-OS highly-granular modularity for improved efficiency and fault isolation
• NX-OS built to compartmentalize, scale (up or down), be portable, and extendable
• Based on proven SAN-OS/IOS & secure/standard features implementation
• Enabling virtual mobility and cloud services
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
NX-OS Software Architecture Summary
91
N
S
Highly Available and Secure
Modular and Efficient
Full-Featured and Cloud Ready
X
O
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Recommended Reading
92
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Additional References
93
• Common Criteria Certification #10349
http://www.niap-ccevs.org/st/vid10349/
• FIPS 140-2
http://csrc.nist.gov/groups/STM/cmvp/documents/140-1/140InProcess.pdf
• NSS Labs
http://www.nsslabs.com/
• Verified Scale Guide on Cisco.com
http://www.cisco.com/en/US/docs/switches/datacenter/sw/verified_scalability/b_Cisco_Nexus_7000_Series_NX-OS_Verified_Scalability_Guide.html
• Follow us on Twitter - @CiscoNexus7000
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Participate in the “My Favorite Speaker” Contest
• Promote your favorite speaker through Twitter and you could win $200 of Cisco Press products (@CiscoPress)
• Send a tweet and include
– Your favorite speaker’s Twitter handle @CCIE5851
– Two hashtags: #CLUS #MyFavoriteSpeaker
• You can submit an entry for more than one of your “favorite” speakers
• Don’t forget to follow @CiscoLive and @CiscoPress
• View the official rules at http://bit.ly/CLUSwin
Promote Your Favorite Speaker and You Could be a Winner
94
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Complete Your Online Session Evaluation
• Give us your feedback and you could win fabulous prizes. Winners announced daily.
• Complete your session evaluation through the Cisco Live mobile app or visit one of the interactive kiosks located throughout the convention center.
Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online
95
© 2014 Cisco and/or its affiliates. All rights reserved. BRKARC-3471 Cisco Public
Continue Your Education
• Demos in the Cisco Campus
• Walk-in Self-Paced Labs
• Table Topics
• Meet the Engineer 1:1 meetings
96