Marian Klas, [email protected]
Systems Engineer – Data Center
February 2015
Integrating Cisco UCS with Cisco ACI
2 © 2015 Cisco and/or its affiliates. All rights reserved.
§ Connecting workloads to ACI § Bare Metal § Hypervisors
§ UCS & APIC Integration and Orchestration
§ Converged Stack Examples
Agenda:
3 © 2015 Cisco and/or its affiliates. All rights reserved.
Connecting workloads to ACI: Bare Metal Servers
4 © 2015 Cisco and/or its affiliates. All rights reserved.
Connecting Physical Workloads to a Port Infrastructure configuration (Access Policy)
4
• Switch 101 • Switch 102 • Etc…
Switch Profile
• Interface 1/1 • Interface 1/2 • Etc…
Interface Profile
• Aggregates port configurations • It can be of type Interface, Port-
channel, vPC etc… Policy Group = port
configuration
5 © 2015 Cisco and/or its affiliates. All rights reserved.
“Enabling” VLANs on a set of ports
5
• Aggregates Domains and has a reference with the policy-group
Attach Entity Profile
• Can be made of multiple VLAN pools Physical Domain
• Defines a range of VLANs VLAN Pool
What is it? It’s just a way to tell the fabric which VLAN ranges are expected where, it’s useful for VLAN reuse
6 © 2015 Cisco and/or its affiliates. All rights reserved.
Mapping between Tenant View and Infrastructure view
Attach Entity Profile
Policy Group = port configuration
Port (i.e. port-profile, switch
profile)
Infrastructure View Tenant View
EPG maps to port, VLAN
VLAN must be part of AEP
8 © 2015 Cisco and/or its affiliates. All rights reserved.
How to Connect Servers
Tenant portchannel leaf
EPG, bridge domain, router
virtu
al m
achi
ne
virtu
al m
achi
ne
9 © 2015 Cisco and/or its affiliates. All rights reserved.
Mapping an EPG to a Server Port
11 © 2015 Cisco and/or its affiliates. All rights reserved.
Endpoints are discovered Under Operational Client Endpoints
12 © 2015 Cisco and/or its affiliates. All rights reserved.
EPG mapped to multiple VLANs on different leafs
Spines
Leafs Border Leafs
vlan10 vlan20 vlan30 vlan40
13 © 2015 Cisco and/or its affiliates. All rights reserved.
ACI Fabric – Endpoint Connectivity Rack Mount Servers
• Direct Connect, FEX Supported – 9300 up to 6 x FEX (as of Jan 2015)
9300 9300 9300 9300
9396PX/TX or 93128TX Leaf
9300 Leaf w/ FEX
N2K N2K
1/10G 40G
• Flexible teaming options including: – Active/Active w/ LACP; – Active/Active with AVS and FEX – Enhanced vPC and vPC w/ FEX not Supported (as of Jan
2015) – Active/Standby
10G
No Host vPC
Policy Enforcement
13
14 © 2015 Cisco and/or its affiliates. All rights reserved.
FEX Topology Support Roadmap
6.1(2)I2(3) Future Future 6.1(2)I2(3)
Straight Through (Single Homed) vPC (Dual Homed) EvPC
Active/Standby Teaming
Nexus 9300 Standalone
Nexus 9300 ACI Leaf
11.1(x) - 1HCY15 11.0(1d) - Shipping Future Future
15 © 2015 Cisco and/or its affiliates. All rights reserved.
FEX Support
• 6.1(2)I2(3) • N2224TP, N2248TP, N2248TP-E,
N2232TM, N2232PP, B22HP
• 6.1(2)I3(1) • B22-Dell, 2232TM-E, 2248PQ FEX
• Q1 CY15 • B22-IBM, B22-Fujitsu, 2348UPQ
• 11.0(1d) - Shipping • N2K-C2248PQ, N2K-C2248TP-E,
N2K-C2248TP-1GE, N2K-C2232PP-10GE, N2K-C2232TM-E
• 11.1(x) - Q1CY15 • B22HP, B22-DEL, B22-IBM
• Q2CY15 • 2348UPQ
Nexus 9300 FEX Support ACI FEX Support
17 © 2015 Cisco and/or its affiliates. All rights reserved.
ACI Fabric – Endpoint Connectivity 3rd Party Blade Servers & Ethernet northbound connectivity
HPVC HPVC
Flex-10 FlexFabric
SW SW
Blade Switches from: • Cisco • HP • Force-10 • IBM/BNT • Blade Servers – HP, IBM, Dell. (Q1CY15 – MR2 release)
• 3rd Party Switches – STP Interoperability • Leaf Port Type – Network/Edge
– Edge Port – BPDU Guard – Network Port – BPDU Pass-Through
B22 B22
B22-HP B22-Dell B22-IBM
N9K N9K N9K N9K N9K N9K N9K N9K
PT PT
Policy Enforcement
L2 Switching
Virtual Switching
No Host vPC
N2K N2K
No Host vPC
ACI Fabric
17
Passthrough
18 © 2015 Cisco and/or its affiliates. All rights reserved.
Connecting workloads to ACI: Hypervisor Integration
19 © 2015 Cisco and/or its affiliates. All rights reserved.
Hypervisors Integration
• General concepts
• Integration with Vmware • Cisco Application Virtual Switch (AVS) • VMware DVS
• Integration with Hyper-V
• Integration with KVM / Openstack
19
20 © 2015 Cisco and/or its affiliates. All rights reserved.
Hypervisor Interaction with ACI • Two modes of Operation
• ACI Fabric as an IP-Ethernet Transport
• Encapsulations manually allocated • Separate Policy domains for Physical
and Virtual
VLAN 10 VLAN 10 VXLAN 10000
Non-Integrated Mode
• ACI Fabric as a Policy Authority • Encapsulations Normalized and
dynamically provisioned • Integrated Policy domains across
Physical and Virtual
APP WEB DB
Integrated Mode
DB
20
21 © 2015 Cisco and/or its affiliates. All rights reserved.
vCenter DVS SCVMM
§ Relationship is formed between APIC and Virtual Machine Manager (VMM)
§ Multiple VMMs likely on a single ACI Fabric
§ Each VMM and associated Virtual hosts are grouped within APIC
§ Called VMM Domain
§ There is 1:1 relationship between a Virtual Switch and VMM Domain
VMM Domain 1
Hypervisor Integration with ACI • Control Channel - VMM Domains
vCenter AVS
VMM Domain 2 VMM Domain 3
21
22 © 2015 Cisco and/or its affiliates. All rights reserved.
L/B
EPGAPP
EPG DB F/W
EPG WEB
Application Network Profile
VM VM VM
WEB PORT GROUP
APP PORT GROUP
DB PORT GROUP
Hypervisor Integration with ACI
APIC § ACI Fabric implements policy on Virtual
Networks by mapping Endpoints to EPGs
§ Endpoints in a Virtualized environment are represented as the vNICs
§ VMM applies network configuration by placement of vNICs into:
§ Port Groups (VMWare), § VM Networks (Hyper-V) § Networks (OpenStack)
§ EPGs are exposed to the VMM as a 1:1 mapping to Port Groups, VM Networks or OpenStack Networking.
22
26 © 2015 Cisco and/or its affiliates. All rights reserved.
Hypervisor Integration with ACI • Endpoint Discovery
DVS Host
APIC
VMM
Control (vCenter API)
Data Path
§ Virtual Endpoints are discovered for reachability & policy purposes via 2 methods:
§ Control Plane Learning: - Out-of-Band Handshake: vCenter APIs
- Inband Handshake: OpFlex-enabled Host (AVS, Hyper-V, etc.)
§ Data Path Learning: Distributed switch learning
§ LLDP/CDP used to resolve Virtual host ID to attached port on leaf node (non-OpFlex Hosts)
OpFlex Host
Control (OpFlex)
Data Path
26
27 © 2015 Cisco and/or its affiliates. All rights reserved.
Hypervisors Integration
• General concepts
• Integration with Vmware • Cisco Application Virtual Switch (AVS) • VMware DVS
• Integration with Hyper-V
• Integration with KVM / Openstack
27
28 © 2015 Cisco and/or its affiliates. All rights reserved.
VMWare Integration • Three Different Options
+
Distributed Virtual Switch (DVS) vCenter + vShield Application Virtual Switch
(AVS)
• Encapsulations: VLAN • Installation: Native • VM discovery: LLDP/
CDP • Software/Licenses:
vCenter with EnterprisePlus License
• Encapsulations: VLAN, VXLAN
• Installation: Native • VM discovery: LLDP/
CDP • Software/Licenses:
vCenter with EnterprisePlus License, vShield Manager with vShield License
• Encapsulations: VLAN, VXLAN
• Installation: VIB through VUM or Console
• VM discovery: OpFlex • Software/Licenses:
vCenter with EnterprisePlus License
28
29 © 2015 Cisco and/or its affiliates. All rights reserved.
APIC Admin
VI/Server Admin Instantiate VMs, Assign to Port Groups
L/B
EPG APP
EPG DB
F/W
EPG WEB
Application Network Profile
Create Application Policy
Web Web Web App
HYPERVISOR HYPERVISOR
VIRTUAL DISTRIBUTED SWITCH
WEB PORT GROUP
APP PORT GROUP
DB PORT GROUP
vCenter Server / vShield
8
5
1
9 ACI Fabric
Automatically Map EPG To Port Groups
Push Policy
Create VDS 2
Cisco APIC and VMware vCenter Initial
Handshake
6
DB DB
7 Create Port Groups
ACI Hypervisor Integration – VMware DVS/vShield
APIC
3
Attach Hypervisor to VDS
4 Learn location of ESX Host through LLDP
29
30 © 2015 Cisco and/or its affiliates. All rights reserved.
ACI Hypervisor Integration – VMware DVS
Name of VMM Domain Type of vSwitch (DVS or AVS) Associated Attachable Entity Profile (AEP) VLAN Pool
vCenter Administrator Credentials
vCenter server information
30
31 © 2015 Cisco and/or its affiliates. All rights reserved.
ACI Hypervisor Integration – VMware DVS
31
33 © 2015 Cisco and/or its affiliates. All rights reserved.
APIC Admin
VI/Server Admin Instantiate VMs, Assign to Port Groups
L/B
EPG APP
EPG DB F/W
EPG WEB
Application Network Profile
Create Application Policy
Web Web Web App
HYPERVISOR HYPERVISOR
Application Virtual Switch (AVS)
WEB PORT GROUP
APP PORT GROUP
DB PORT GROUP
vCenter Server
8
5
1
9 ACI Fabric
Automatically Map EPG To Port Groups
Push Policy
Create AVS VDS 2
Cisco APIC and VMware vCenter Initial
Handshake
6
DB DB
7 Create Port Groups
ACI Hypervisor Integration – AVS
APIC
3
Attach Hypervisor to VDS
4 Learn location of ESX Host through OpFlex
OpFlex Agent OpFlex Agent
33
34 © 2015 Cisco and/or its affiliates. All rights reserved.
§ AVS supports OpFlex to integrate with APIC
§ Supports a Full multi-hop Layer 2 Network between Nexus 9k and AVS: Investment Protection
§ VMware DVS can only support a single L2 switch between N9k and DVS § LLDP/CDP and NOT OpFlex
Integration
§ Layer 2 network is required to support OpFlex bootstrapping in this phase
Extending ACI to Existing Virtual & Physical Network
Layer 2 Network
AVS
AVS
AVS O
pFlex
OpFlex
OpFlex
Phase 1: Layer 2 Existing Network/Local Switching
36 © 2015 Cisco and/or its affiliates. All rights reserved.
Hypervisors Integration
• General concepts
• Integration with Vmware • Cisco Application Virtual Switch (AVS) • VMware DVS
• Integration with Hyper-V
• Integration with KVM / Openstack
36
37 © 2015 Cisco and/or its affiliates. All rights reserved.
Microsoft Interaction with ACI • Two modes of Operation
• Policy Management: Through APIC • Software / License: Windows Server with
HyperV, SCVMM • VM Discovery: OpFlex • Encapsulations: VLAN, NVGRE (Future) • Plugin Installation: Manual
Integration with SCVMM
APIC
Integration with Azure Pack
APIC
• Superset of SCVMM • Policy Management: Through APIC or
through Azure Pack • Software / License: Windows Server with
HyperV, SCVMM, Azure Pack (free) • VM Discovery: OpFlex • Encapsulations: VLAN, NVGRE (Future) • Plugin Installation: Integrated
+
37
39 © 2015 Cisco and/or its affiliates. All rights reserved.
Microsoft Azure Pack Integration
§ Integration with Microsoft requires: - Windows Server 2012 - Systems Center 2012 R2 with SPF - Windows Azure Pack
§ Azure Pack provides single pane of glass for Definition, creation, management of their cloud service
§ Divided into Provider (Admin) portal and Consumer Self-Service (Tenant) portal
§ Cisco ACI Service Plugin enables management of Network Infrastructure through APIC REST API
R2 w/ Service Provider Foundation
Web Sites
Service Plans Users
Provider Portal
Consumer Self-Service
Portal
Web Sites Apps Database VMs ACI
Service Provider Customer
VMs SQL Service Bus …
39
41 © 2015 Cisco and/or its affiliates. All rights reserved.
APIC Admin (Basic Infrastructure)
Azure Pack Tenant
3
6
ACI Fabric
Push Network Profiles to APIC
Pull Policy on leaf where EP attaches
Indicate EP Attach to attached leaf when VM starts
1
2
HYPERVISOR HYPERVISOR HYPERVISOR
ACI Azure Pack Integration
APIC
Get VLANs allocated for each EPG
Create Application Policy
7
Azure Pack \ SPF
SCVMM Plugin APIC Plugin OpFlex Agent OpFlex Agent OpFlex Agent
Instantiate VMs
5
1
4Create VM Networks
4
41
Web Web Web Web App App DB DB
42 © 2015 Cisco and/or its affiliates. All rights reserved.
Hypervisors Integration
• General concepts
• Integration with Vmware • Cisco Application Virtual Switch (AVS) • VMware DVS
• Integration with Hyper-V
• Integration with KVM / Openstack
42
46 © 2015 Cisco and/or its affiliates. All rights reserved.
APIC Admin (Performs Steps 3)
OpenStack Tenant (Performs Steps 1,4) Instantiate VMs
Create Application Policy
Web Web Web Web App App 4
3
5 ACI Fabric
Automatically Push Network Profiles to APIC
Push Policy
Create Network, Subnet, Security Groups, Policy
NETWORK
ROUTING
SECURITY
1
2
DB DB
HYPERVISOR HYPERVISOR HYPERVISOR
NOVA
NEUTRON
OPEN VIRTUAL SWITCH OPEN VIRTUAL SWITCH OPEN VIRTUAL SWITCH
ACI OpenStack Integration – Phase 1
APIC
46
47 © 2015 Cisco and/or its affiliates. All rights reserved.
ACI OpenStack Integration – Phase 2 (Group-based policy)
2
ACI Admin (manages physical
network, monitors tenant state)
L/B
EPG APP
EPG DB F/W
L/B
EPG WEB
Application Network Profile
Create Application Policy
3
5 ACI Fabric
Push Policy
OpenStack Tenant (Performs step 1,4) Instantiate VMs
Web Web Web Web App App 4
Create Application Network Profile
1
DB DB
HYPERVISOR HYPERVISOR HYPERVISOR
NOVA
NEUTRON
Automatically Push Network Profiles to APIC
L/B
EPG APP
EPG DB F/W
L/B
EPG WEB
Application Network Profile
APIC
48 © 2015 Cisco and/or its affiliates. All rights reserved.
UCS & APIC Integration
51 © 2015 Cisco and/or its affiliates. All rights reserved.
ACI Management Overview with UCS End-Host Mode SAN A SAN B
Ethernet Ethernet and FCOE
FC FCOE
UCSM / UCS Central
UCS Director
51
52 © 2015 Cisco and/or its affiliates. All rights reserved. Ethernet Ethernet and
FCOE
FC FCOE
UCS Director
UCSM/ UCS Central
EPG
SP Binding - VLANs
UCS Stand-alone
SAN A SAN B
UCSM / UCS Central
UCS Director
ACI Management Overview with UCS End-Host Mode
52
57 © 2015 Cisco and/or its affiliates. All rights reserved.
UCS Director
Storage
A B C Virtualization Baremetal
Compute
Server Load Balancer
L4-7 Services
Application Catalog Create SharePoint
App Policy in APIC
Create Additional Storage for DB Tier
VM VM
VM VM VM
VM
VM
VM
VM
Web
App
DB
L4-7
L4-7
Application Fully Instantiated
Storage Manager
Create VMs Assign to EPGs Assign Baremetal Servers to EPGs
Fabric
UCSD Application Provisioning Lifecycle
58 © 2015 Cisco and/or its affiliates. All rights reserved.
§ Consistent Virtualized and Physical Server Deployment Models
§ Multi-Phase Approach § All phases show a Cisco together value add § First 2 on existing UCS deployments § Later one on 3rd Generation UCS FI
§ Phase 1 (2014): UCS Director to orchestrate workloads over interlinked UCS and ACI
§ Phase 2 (2015): UCS and ACI sharing policy and state for a better together story
§ Phase 3 (Planning): 3rd Gen UCS Fabric Interconnect is a leaf with expanded policy and state interaction
Common Policy Based Infrastructure Programmability Flexibility, Performance, and Visibility
Automated Configuration via
UCS Director
Policy and State Exchange via
Software Agent
Integrated Management as Native ACI Leaf
UCS FI’s
61 © 2015 Cisco and/or its affiliates. All rights reserved.
Phases and Tradeoffs – UCSM Managed Servers Feature Phase 1 Phase 2a Phase 2b Phase 3
UCS Fabric Interconnects
6100/6200 6100/6200 6300 6300
UCS to ACI Leaf vPC Pair Ratio
N:1 (10GE) N:1 (10GE) N:1 (40GE) 1:1 (Can be a Leaf)
Port Types/Speeds 10GE SFP, 8GFC SFP 10GE SFP, 8GFC SFP 40GE QSFP, 16GFC SFP
40GE SFP, 16GFC SFP
UCS Manager Location Within FI Within FI Within FI Decoupled and containerized
Policy Integration Agent UCS Director Over the Top Bridge OpFlex Control Path OpFlex Control Path between DME’s
UCS Physical Attachment
Connected to ACI Leaf Connected to ACI Leaf Connected to ACI Leaf Is ACI Leaf – Connected to ACI Spine
Health Scoring To ACI Leaf Intra UCS fabric data feed to Observer
Inside OpFlex Path Direct from UCS FI ASIC and OS
End Point Group Mechanisms
Construction of EPG to VLAN within UCSD
Auto-Population of EPG to VLAN data
Auto-Population of EPG to VLAN data
EPG assignment and encapsulation per UCS server interface
62 © 2015 Cisco and/or its affiliates. All rights reserved.
Converged stack examples
63 © 2015 Cisco and/or its affiliates. All rights reserved.
ACI-ready Vblock and FlexPOD
63
http://newsroom.cisco.com/press-release-content?type=webcontent&articleId=1421361
64 © 2015 Cisco and/or its affiliates. All rights reserved.
FlexPod with ACI - Cisco and Partner Technologies
Key Cisco and Partner Technologies Unified Compute System Blade and Rack mount Servers Nexus 9000 Top of Rack and Modular switches NetApp Clustered Data ONTAP utilizing storage virtual machines VMware Hypervisor and vCenter management Firewall and Load Balancer Services
Key Components of FlexPod with ACI design
§ Nexus 9500/9300 Spine & Leaf Switches
§ 3 node Cisco Application Policy Infrastructure Controller (APIC)
§ UCS Manager 2.2 release
§ vSphere 5.5 update 2
§ L4-L7 Services, ASA Firewall and F5 Load balancer
§ NetApp FAS 8000 and CDOT 8.2.1
§ OnCommand System Manager
§ NetApp Snap Manager
Application Policy Infrastructure Controller
Hypervisor
Hypervisor Managers
Integrated L4-L7
Services
Load Balancer
NetApp Clustered ONTAP
ACI Fabric / Nexus 9000 spine / leaf
Cisco Unified Computing
System
APIC
65 © 2015 Cisco and/or its affiliates. All rights reserved.
FlexPod Data Center with ACI
§ FlexPod Data Center pre-validated Integration with ACI
§ Configuration management using GUI in the current release – UCS Director to be incorporated in upcoming designs
§ Application Validation § Microsoft Exchange 2013 § Microsoft SharePoint 2013 § Microsoft SQL Server 2012 SP1
§ Additional Applications to be validated in future
Thank you.