integrating cisco ucs with cisco aci · ucs & apic integration and orchestration ! ... l/b epg...

Marian Klas, [email protected]

Systems Engineer – Data Center

February 2015

Integrating Cisco UCS with Cisco ACI

2 © 2015 Cisco and/or its affiliates. All rights reserved.

§  Connecting workloads to ACI §  Bare Metal §  Hypervisors

§  UCS & APIC Integration and Orchestration

§  Converged Stack Examples

Agenda:


Connecting workloads to ACI: Bare Metal Servers


Connecting Physical Workloads to a Port Infrastructure configuration (Access Policy)

4

• Switch 101 • Switch 102 • Etc…

Switch Profile

•  Interface 1/1 •  Interface 1/2 • Etc…

Interface Profile

• Aggregates port configurations •  It can be of type Interface, Port-

channel, vPC etc… Policy Group = port

configuration


“Enabling” VLANs on a set of ports

5

• Aggregates Domains and has a reference with the policy-group

Attach Entity Profile

• Can be made of multiple VLAN pools Physical Domain

• Defines a range of VLANs VLAN Pool

What is it? It’s just a way to tell the fabric which VLAN ranges are expected where, it’s useful for VLAN reuse


Mapping between Tenant View and Infrastructure view

Attach Entity Profile

Policy Group = port configuration

Port (i.e. port-profile, switch

profile)

Infrastructure View Tenant View

EPG maps to port, VLAN

VLAN must be part of AEP


How to Connect Servers

Tenant portchannel leaf

EPG, bridge domain, router

virtu

al m

achi

ne

virtu

al m

achi

ne


Mapping an EPG to a Server Port


Endpoints are discovered Under Operational Client Endpoints


EPG mapped to multiple VLANs on different leafs

Spines

Leafs Border Leafs

vlan10 vlan20 vlan30 vlan40


ACI Fabric – Endpoint Connectivity Rack Mount Servers

•  Direct Connect, FEX Supported –  9300 up to 6 x FEX (as of Jan 2015)

9300 9300 9300 9300

9396PX/TX or 93128TX Leaf

9300 Leaf w/ FEX

N2K N2K

1/10G 40G

•  Flexible teaming options including: –  Active/Active w/ LACP; –  Active/Active with AVS and FEX –  Enhanced vPC and vPC w/ FEX not Supported (as of Jan

2015) –  Active/Standby

10G

No Host vPC

Policy Enforcement

13


FEX Topology Support Roadmap

6.1(2)I2(3) Future Future 6.1(2)I2(3)

Straight Through (Single Homed) vPC (Dual Homed) EvPC

Active/Standby Teaming

Nexus 9300 Standalone

Nexus 9300 ACI Leaf

11.1(x) - 1HCY15 11.0(1d) - Shipping Future Future


FEX Support

•  6.1(2)I2(3) •  N2224TP, N2248TP, N2248TP-E,

N2232TM, N2232PP, B22HP

•  6.1(2)I3(1) •  B22-Dell, 2232TM-E, 2248PQ FEX

•  Q1 CY15 •  B22-IBM, B22-Fujitsu, 2348UPQ

•  11.0(1d) - Shipping •  N2K-C2248PQ, N2K-C2248TP-E,

N2K-C2248TP-1GE, N2K-C2232PP-10GE, N2K-C2232TM-E

•  11.1(x) - Q1CY15 •  B22HP, B22-DEL, B22-IBM

•  Q2CY15 •  2348UPQ

Nexus 9300 FEX Support ACI FEX Support


ACI Fabric – Endpoint Connectivity 3rd Party Blade Servers & Ethernet northbound connectivity

HPVC HPVC

Flex-10 FlexFabric

SW SW

Blade Switches from: •  Cisco •  HP •  Force-10 •  IBM/BNT •  Blade Servers – HP, IBM, Dell. (Q1CY15 – MR2 release)

•  3rd Party Switches – STP Interoperability •  Leaf Port Type – Network/Edge

–  Edge Port – BPDU Guard –  Network Port – BPDU Pass-Through

B22 B22

B22-HP B22-Dell B22-IBM

N9K N9K N9K N9K N9K N9K N9K N9K

PT PT

Policy Enforcement

L2 Switching

Virtual Switching

No Host vPC

N2K N2K

No Host vPC

ACI Fabric

17

Passthrough


Connecting workloads to ACI: Hypervisor Integration


Hypervisors Integration

•  General concepts

•  Integration with Vmware •  Cisco Application Virtual Switch (AVS) •  VMware DVS

•  Integration with Hyper-V

•  Integration with KVM / Openstack

19


Hypervisor Interaction with ACI •  Two modes of Operation

•  ACI Fabric as an IP-Ethernet Transport

•  Encapsulations manually allocated •  Separate Policy domains for Physical

and Virtual

VLAN 10 VLAN 10 VXLAN 10000

Non-Integrated Mode

•  ACI Fabric as a Policy Authority •  Encapsulations Normalized and

dynamically provisioned •  Integrated Policy domains across

Physical and Virtual

APP WEB DB

Integrated Mode

DB

20


vCenter DVS SCVMM

§  Relationship is formed between APIC and Virtual Machine Manager (VMM)

§  Multiple VMMs likely on a single ACI Fabric

§  Each VMM and associated Virtual hosts are grouped within APIC

§  Called VMM Domain

§  There is 1:1 relationship between a Virtual Switch and VMM Domain

VMM Domain 1

Hypervisor Integration with ACI •  Control Channel - VMM Domains

vCenter AVS

VMM Domain 2 VMM Domain 3

21


L/B

EPGAPP

EPG DB F/W

EPG WEB

Application Network Profile

VM VM VM

WEB PORT GROUP

APP PORT GROUP

DB PORT GROUP

Hypervisor Integration with ACI

APIC §  ACI Fabric implements policy on Virtual

Networks by mapping Endpoints to EPGs

§  Endpoints in a Virtualized environment are represented as the vNICs

§  VMM applies network configuration by placement of vNICs into:

§  Port Groups (VMWare), §  VM Networks (Hyper-V) §  Networks (OpenStack)

§  EPGs are exposed to the VMM as a 1:1 mapping to Port Groups, VM Networks or OpenStack Networking.

22


Hypervisor Integration with ACI •  Endpoint Discovery

DVS Host

APIC

VMM

Control (vCenter API)

Data Path

§  Virtual Endpoints are discovered for reachability & policy purposes via 2 methods:

§  Control Plane Learning: -  Out-of-Band Handshake: vCenter APIs

-  Inband Handshake: OpFlex-enabled Host (AVS, Hyper-V, etc.)

§  Data Path Learning: Distributed switch learning

§  LLDP/CDP used to resolve Virtual host ID to attached port on leaf node (non-OpFlex Hosts)

OpFlex Host

Control (OpFlex)

Data Path

26







27


VMWare Integration •  Three Different Options

+

Distributed Virtual Switch (DVS) vCenter + vShield Application Virtual Switch

(AVS)

•  Encapsulations: VLAN •  Installation: Native •  VM discovery: LLDP/

CDP •  Software/Licenses:

vCenter with EnterprisePlus License

•  Encapsulations: VLAN, VXLAN

•  Installation: Native •  VM discovery: LLDP/

CDP •  Software/Licenses:

vCenter with EnterprisePlus License, vShield Manager with vShield License

•  Encapsulations: VLAN, VXLAN

•  Installation: VIB through VUM or Console

•  VM discovery: OpFlex •  Software/Licenses:

vCenter with EnterprisePlus License

28


APIC Admin

VI/Server Admin Instantiate VMs, Assign to Port Groups

L/B

EPG APP

EPG DB

F/W

EPG WEB


Create Application Policy

Web Web Web App

HYPERVISOR HYPERVISOR

VIRTUAL DISTRIBUTED SWITCH

WEB PORT GROUP

APP PORT GROUP

DB PORT GROUP

vCenter Server / vShield

8

5

1

9 ACI Fabric

Automatically Map EPG To Port Groups

Push Policy

Create VDS 2

Cisco APIC and VMware vCenter Initial

Handshake

6

DB DB

7 Create Port Groups

ACI Hypervisor Integration – VMware DVS/vShield

APIC

3

Attach Hypervisor to VDS

4 Learn location of ESX Host through LLDP

29


ACI Hypervisor Integration – VMware DVS

Name of VMM Domain Type of vSwitch (DVS or AVS) Associated Attachable Entity Profile (AEP) VLAN Pool

vCenter Administrator Credentials

vCenter server information

30


ACI Hypervisor Integration – VMware DVS

31


APIC Admin

VI/Server Admin Instantiate VMs, Assign to Port Groups

L/B

EPG APP

EPG DB F/W

EPG WEB



Web Web Web App

HYPERVISOR HYPERVISOR

Application Virtual Switch (AVS)

WEB PORT GROUP

APP PORT GROUP

DB PORT GROUP

vCenter Server

8

5

1

9 ACI Fabric

Automatically Map EPG To Port Groups

Push Policy

Create AVS VDS 2

Cisco APIC and VMware vCenter Initial

Handshake

6

DB DB

7 Create Port Groups

ACI Hypervisor Integration – AVS

APIC

3

Attach Hypervisor to VDS

4 Learn location of ESX Host through OpFlex

OpFlex Agent OpFlex Agent

33


§  AVS supports OpFlex to integrate with APIC

§  Supports a Full multi-hop Layer 2 Network between Nexus 9k and AVS: Investment Protection

§  VMware DVS can only support a single L2 switch between N9k and DVS §  LLDP/CDP and NOT OpFlex

Integration

§  Layer 2 network is required to support OpFlex bootstrapping in this phase

Extending ACI to Existing Virtual & Physical Network

Layer 2 Network

AVS

AVS

AVS O

pFlex

OpFlex

OpFlex

Phase 1: Layer 2 Existing Network/Local Switching







36


Microsoft Interaction with ACI •  Two modes of Operation

•  Policy Management: Through APIC •  Software / License: Windows Server with

HyperV, SCVMM •  VM Discovery: OpFlex •  Encapsulations: VLAN, NVGRE (Future) •  Plugin Installation: Manual

Integration with SCVMM

APIC

Integration with Azure Pack

APIC

•  Superset of SCVMM •  Policy Management: Through APIC or

through Azure Pack •  Software / License: Windows Server with

HyperV, SCVMM, Azure Pack (free) •  VM Discovery: OpFlex •  Encapsulations: VLAN, NVGRE (Future) •  Plugin Installation: Integrated

+

37


Microsoft Azure Pack Integration

§  Integration with Microsoft requires: -  Windows Server 2012 -  Systems Center 2012 R2 with SPF -  Windows Azure Pack

§  Azure Pack provides single pane of glass for Definition, creation, management of their cloud service

§  Divided into Provider (Admin) portal and Consumer Self-Service (Tenant) portal

§  Cisco ACI Service Plugin enables management of Network Infrastructure through APIC REST API

R2 w/ Service Provider Foundation

Web Sites

Service Plans Users

Provider Portal

Consumer Self-Service

Portal

Web Sites Apps Database VMs ACI

Service Provider Customer

VMs SQL Service Bus …

39


APIC Admin (Basic Infrastructure)

Azure Pack Tenant

3

6

ACI Fabric

Push Network Profiles to APIC

Pull Policy on leaf where EP attaches

Indicate EP Attach to attached leaf when VM starts

1

2

HYPERVISOR HYPERVISOR HYPERVISOR

ACI Azure Pack Integration

APIC

Get VLANs allocated for each EPG


7

Azure Pack \ SPF

SCVMM Plugin APIC Plugin OpFlex Agent OpFlex Agent OpFlex Agent

Instantiate VMs

5

1

4Create VM Networks

4

41

Web Web Web Web App App DB DB







42


APIC Admin (Performs Steps 3)

OpenStack Tenant (Performs Steps 1,4) Instantiate VMs


Web Web Web Web App App 4

3

5 ACI Fabric

Automatically Push Network Profiles to APIC

Push Policy

Create Network, Subnet, Security Groups, Policy

NETWORK

ROUTING

SECURITY

1

2

DB DB


NOVA

NEUTRON

OPEN VIRTUAL SWITCH OPEN VIRTUAL SWITCH OPEN VIRTUAL SWITCH

ACI OpenStack Integration – Phase 1

APIC

46


ACI OpenStack Integration – Phase 2 (Group-based policy)

2

ACI Admin (manages physical

network, monitors tenant state)

L/B

EPG APP

EPG DB F/W

L/B

EPG WEB



3

5 ACI Fabric

Push Policy

OpenStack Tenant (Performs step 1,4) Instantiate VMs

Web Web Web Web App App 4

Create Application Network Profile

1

DB DB


NOVA

NEUTRON

Automatically Push Network Profiles to APIC

L/B

EPG APP

EPG DB F/W

L/B

EPG WEB


APIC


UCS & APIC Integration


ACI Management Overview with UCS End-Host Mode SAN A SAN B

Ethernet Ethernet and FCOE

FC FCOE

UCSM / UCS Central

UCS Director

51

52 © 2015 Cisco and/or its affiliates. All rights reserved. Ethernet Ethernet and

FCOE

FC FCOE

UCS Director

UCSM/ UCS Central

EPG

SP Binding - VLANs

UCS Stand-alone

SAN A SAN B

UCSM / UCS Central

UCS Director

ACI Management Overview with UCS End-Host Mode

52


UCS Director

Storage

A B C Virtualization Baremetal

Compute

Server Load Balancer

L4-7 Services

Application Catalog Create SharePoint

App Policy in APIC

Create Additional Storage for DB Tier

VM VM

VM VM VM

VM

VM

VM

VM

Web

App

DB

L4-7

L4-7

Application Fully Instantiated

Storage Manager

Create VMs Assign to EPGs Assign Baremetal Servers to EPGs

Fabric

UCSD Application Provisioning Lifecycle


§  Consistent Virtualized and Physical Server Deployment Models

§  Multi-Phase Approach §  All phases show a Cisco together value add §  First 2 on existing UCS deployments §  Later one on 3rd Generation UCS FI

§  Phase 1 (2014): UCS Director to orchestrate workloads over interlinked UCS and ACI

§  Phase 2 (2015): UCS and ACI sharing policy and state for a better together story

§  Phase 3 (Planning): 3rd Gen UCS Fabric Interconnect is a leaf with expanded policy and state interaction

Common Policy Based Infrastructure Programmability Flexibility, Performance, and Visibility

Automated Configuration via

UCS Director

Policy and State Exchange via

Software Agent

Integrated Management as Native ACI Leaf

UCS FI’s


Phases and Tradeoffs – UCSM Managed Servers Feature Phase 1 Phase 2a Phase 2b Phase 3

UCS Fabric Interconnects

6100/6200 6100/6200 6300 6300

UCS to ACI Leaf vPC Pair Ratio

N:1 (10GE) N:1 (10GE) N:1 (40GE) 1:1 (Can be a Leaf)

Port Types/Speeds 10GE SFP, 8GFC SFP 10GE SFP, 8GFC SFP 40GE QSFP, 16GFC SFP

40GE SFP, 16GFC SFP

UCS Manager Location Within FI Within FI Within FI Decoupled and containerized

Policy Integration Agent UCS Director Over the Top Bridge OpFlex Control Path OpFlex Control Path between DME’s

UCS Physical Attachment

Connected to ACI Leaf Connected to ACI Leaf Connected to ACI Leaf Is ACI Leaf – Connected to ACI Spine

Health Scoring To ACI Leaf Intra UCS fabric data feed to Observer

Inside OpFlex Path Direct from UCS FI ASIC and OS

End Point Group Mechanisms

Construction of EPG to VLAN within UCSD

Auto-Population of EPG to VLAN data

Auto-Population of EPG to VLAN data

EPG assignment and encapsulation per UCS server interface


Converged stack examples


ACI-ready Vblock and FlexPOD

63

http://newsroom.cisco.com/press-release-content?type=webcontent&articleId=1421361


FlexPod with ACI - Cisco and Partner Technologies

Key Cisco and Partner Technologies Unified Compute System Blade and Rack mount Servers Nexus 9000 Top of Rack and Modular switches NetApp Clustered Data ONTAP utilizing storage virtual machines VMware Hypervisor and vCenter management Firewall and Load Balancer Services

Key Components of FlexPod with ACI design

§  Nexus 9500/9300 Spine & Leaf Switches

§  3 node Cisco Application Policy Infrastructure Controller (APIC)

§  UCS Manager 2.2 release

§  vSphere 5.5 update 2

§  L4-L7 Services, ASA Firewall and F5 Load balancer

§  NetApp FAS 8000 and CDOT 8.2.1

§  OnCommand System Manager

§  NetApp Snap Manager

Application Policy Infrastructure Controller

Hypervisor

Hypervisor Managers

Integrated L4-L7

Services

Load Balancer

NetApp Clustered ONTAP

ACI Fabric / Nexus 9000 spine / leaf

Cisco Unified Computing

System

APIC


FlexPod Data Center with ACI

§  FlexPod Data Center pre-validated Integration with ACI

§  Configuration management using GUI in the current release – UCS Director to be incorporated in upcoming designs

§  Application Validation §  Microsoft Exchange 2013 §  Microsoft SharePoint 2013 §  Microsoft SQL Server 2012 SP1

§  Additional Applications to be validated in future

Thank you.

integrating cisco ucs with cisco aci · ucs & apic integration and orchestration ! ... l/b epg...

Documents