virtual machine fabric extender (vm-fex)...

© 2012 Cisco and/or its affiliates. All rights reserved. BRKCOM-2005 Cisco Public

Virtual Machine Fabric Extension (VM-FEX) Bringing the Virtual Machines Directly on the Network

BRKCOM-2005

Dan Hanson, Technical Marketing Manager, Data Center Group, CCIE #4482

2


Agenda

Fabric Extension Technology Overview

Virtual Machine Fabric Extender (VM-FEX) Introduction

VM-FEX Operational Model

VM-FEX General Details on UCS and Nexus 5500

VM-FEX Implementation with VMware on UCS and Nexus 5500

VM-FEX Implementation with KVM on UCS

VM-FEX Implementation with Hyper-V on UCS

Easy-VMFEX tool usage

Performance Review

Summary

What This Session Will Cover

3

Fabric Extension Technology Overview


Fabric Extension (FEX) Concept Virtualising the Network Port

LAN LAN Switch port extended over Fabric Extender

Lo

gic

al S

wit

ch

Switch

Switch

Legacy multi-tier architecture FEX architecture

Switch

FEX

Collapse network tiers, fewer network management points 5


Nexus 5000 + Fabric Extender Single Access Layer

=

Distributed Modular System

+

Nexus 5000 Parent Switch

Cisco Nexus® 2000 FEX

Over 6000 production customers Over 5 million Nexus 2000 ports deployed

Distributed Modular System Nexus 2000 FEX is a Virtual Line Card to the Nexus 5000

Nexus 5000 maintains all management & configuration

No Spanning Tree between FEX & Nexus 5000

LAN

N7000/ C6500

MDS

SAN

Access Layer N5000

1 12

N2232 N2232

6


VNTAG mimics forwarding vectors inside a switch

D: Direction, P: Unicast/Multicast, L: Loop

Policy associated with the Virtual Interface NOT port

VLAN member ship, QoS, MTU, Rate limit etc

VNTAG Ether type

Destination Virtual Interface

Source Virtual Interface ver

P

R

Application Payload

TCP IP

Ethernet VNTAG

FEX architecture

Switch

FEX

LAN

Frame

VNTAG Frame

Key Architectural Component #1: VNTAG “Intra-Chassis” Bus Header

L

D

7


FEX Data Forwarding

Constellation Bus had 32 byte header for fabric switching ‒ Vast majority of modular switch vendors have an internal “Tag” for fabric

communications

Originally, Centralized forwarding ASICs ‒ Line cards fed into these ASICs directly

When we needed higher performance – we added faster Switch Fabrics, and Distributed Forwarding Capabilities to system

What this really meant – adding more ASIC forwarding capacity to the system to minimize the number of devices a flow had to traverse

Revisiting Traditional Modular Switches (Example Catalyst 6500)

8


FEX Data Forwarding

Think the original C6k Satellite Program for VSL and RSL

The Constellation Bus now is smaller header – 6 Byte VNtag header

‒ Core to FEX technology and being standardized as 802.1BR

‒ This is NOT a 1:1 mapping to VEPA/802.1bg which is designed to offer an enhanced forwarding mechanism between peer devices via a single upstream device

Keep the ASIC counts for high performance but put them on the Central controlling switch instead of all these line cards

‒ Latency and bandwidth were more a function of the layers of ASICs to traverse in a tree – rather than the location of these ASICs (the fiber/copper paths for a packet to propagate)

Add protocols for configuration and firmware management of these remote cards (Satellite Control Protocol, Satellite Discovery Protocol)

‒ Allows us to get away from manual firmware code management per (remote) line-card

Move from Store-and-Forward behavior to Cut-Through switching to make latency actually better

Decoupling the Modular Switch

9


LAN

Parent Switch

SAN

FEX

Parent Switch + FEX Decoupling the Modular Switch

10


FEX Technology for Unified I/O

Virtual Switch Ports, Cables, and NIC Ports

Mapping of Ethernet and FC Wires over Ethernet

Service Level enforcement Multiple data types (jumbo, lossless,

FC)

Individual link-states Fewer Cables

Multiple Ethernet traffic co-exist on same cable

Fewer adapters needed Overall less power Interoperates with existing Models

Management remains constant for system admins and LAN/SAN admins

Possible to take these links further upstream for aggregation

Individual Ethernets

DCB Ethernet

Individual Storage (iSCSI, NFS, FC)

Blade Management Channels (KVM, USB, CDROM, Adapters)

11

Virtual Machine Fabric Extender (VM-FEX) Introduction


Extending FEX Architecture to the VMs Cascading of Fabric Extenders

Lo

gic

al S

wit

ch

Virtualized Deployment

Switch

FEX

Hypervisor vSwitch

App

OS

App

OS

App

OS

LAN

Lo

gic

al S

wit

ch

VM-FEX architecture

Switch

FEX

Hypervisor

LAN

App

OS

App

OS

App

OS

VM-FEX

Switch port extended over cascaded Fabric Extenders to the Virtual Machine

L

og

ical

Sw

itch

13


Key Architectural Component #2: UCS VIC UCS Virtual Interface Card Family

256 PCIe devices

Devices can be vNICs or vHBAs

Each device has a corresponding switch interface

Bandwidth 2x4x10 Gb

Uses 4x10 Ether Channel, HW 40Gb Capable

vNICs/vHBAs NOT limited to 10Gb

PCIe Gen-2 x 16

Mezzanine and PCIe

14


UCS VM-FEX Distributed Modular System Removing the Virtual Switching Infrastructure to a FEX

=


VM-FEX: Single Virtual-Physical Access Layer Collapse virtual and physical switching into a single access layer

VM-FEX is a Virtual Line Card to the parent switch

Parent switch maintains all management & configuration

Virtual and Physical traffic treated the same

LAN

N7000/ C6500

MDS

SAN

Access Layer UCS 6100

1 160

UCS VIC UCS VIC

App

OS

App

OS

App

OS App

OS

App

OS

App

OS

App

OS

App

OS

App

OS App

OS

App

OS

App

OS

UCS IOM UCS IOM

+

UCS Fabric Interconnect Parent Switch

Cisco UCS VIC

UCS IOM-FEX

+

15


Nexus 5000/2000 VM-FEX Distributed Modular System Removing the Virtual Switching Infrastructure to a FEX

=


VM-FEX: Single Virtual-Physical Access Layer Collapse virtual and physical switching into a single access layer

VM-FEX is a Virtual Line Card to the parent switch

Parent switch maintains all management & configuration

Virtual and Physical traffic treated the same

LAN

N7000/ C6500

MDS

SAN

Access Layer Nexus 5500

1 160

UCS VIC UCS VIC

App

OS

App

OS

App

OS App

OS

App

OS

App

OS

App

OS

App

OS

App

OS App

OS

App

OS

App

OS

Nexus 2000 Nexus 2000

+

Nexus 5500 Parent Switch

Cisco UCS VIC

Nexus 2000 FEX

+

16


Software Based Virtual Access Layer Out of the Box Deployments

Physical Network

Virtual Network

Hyp

ervi

sor

Hyp

ervi

sor

VM VM VM VM VM VM VM VM

VETH

VNIC

18


VM-FEX Operations Model

Step 1: Preboot

‒ UCS defined PCIe devices and enumerations

‒ Host discovers PCIe devices

Pre-Boot Configuration

Hyp

ervi

sor

Hyp

ervi

sor

19



Step 1: Preboot



Step 2: Port Profile

‒ Folder of Network Policy defined

Defining “Port Profiles” on the UCS or Nexus 5000

Hyp

ervi

sor

Hyp

ervi

sor

Port Profiles Definition

WEB Apps

HR

DB

Compliance

VLAN Web VLAN HR

VLAN DB VLAN Comp

UCSM or Nexus 5500

20



Step 1: Preboot




‒ Folder of Network Policy on UCS or Nexus 5500 defined

Step 3: Port Profile Export

‒ Port Profile name list exported to virtualization manager

Pushing Port Profiles to the Hypervisor System

Hyp

ervi

sor

Hyp

ervi

sor

VLAN Web VLAN HR

VLAN DB VLAN Comp

Hypervisor Manager

UCSM or Nexus 5500 exports Port Profiles

UCSM or Nexus 5500

21



Step 1: Preboot




‒ Folder of Network Policy on UCS or Nexus 5500 defined

Step 3: Port Profile Export

‒ Port Profile name list exported to virtualization manager

Step 4: VM Definition

‒ Named Policy in VM

Mapping of Port Profiles to VM Virtual Adapters

Hyp

ervi

sor

Hyp

ervi

sor

VLAN Web VLAN HR

VLAN DB VLAN Comp

Hypervisor Manager

Network Manager

VM

VM VM VM

UCS or Nexus 5500 exports Port Profiles

22


VM-FEX Modes of Operation Enumeration vs. Hypervisor Bypass

Emulated Mode PCIe Pass-Thru or VMDirectPath

Standard (Emulated) Mode Each VM gets a dedicated PCIe

device

~12%-15% CPU performance improvement

Appears as distributed virtual switch to hypervisor

LiveMigration supported

High Performance Mode

Co-exists with Standard mode

Bypasses Hypervisor layer

~30% improvement in I/O performance

Appears as distributed virtual switch to hypervisor

Currently supported with ESX 5.0 only

vMotion supported

23


VM-FEX Operational Model vMotion with Hypervisor Bypass (VMDirectPath with VM-FEX)

Temporary transition from VMDirectPath to

standard I/O

• VM Sending TCP stream (1500MTU) • UCS B200 M2 blades with UCS VIC card

0

2500

5000

7500

10000

19:0

6:19

19:0

6:23

19:0

6:27

19:0

6:31

19:0

6:35

19:0

6:39

19:0

6:43

19:0

6:47

19:0

6:52

Mb

ps

Time (secs)

vMotion to secondary host

1 sec silent period

24


VM-FEX Operational Model Simplifying the Access Infrastructure

Physical Network

Virtual Network

Hyp

ervi

sor

Hyp

ervi

sor


VETH

VNIC

Unify the virtual and physical network ‒ Same Port Profiles for various

hypervisors and bare metal servers

Consistent functions, performance, management

25



Programmable Access Layer ‒ Basic Starting Points

Software Definable Networking (SDN)

Standardize Troubleshooting ‒ SPAN usage, Performance Trending

Traffic Engineering

Traffic Management Physical Network

Virtual Network


VLANs

26



Removing performance dependencies from VM location

Offloading software switching functionalities from host CPU

More on this in upcoming slides

Traffic Forwarding

Physical Network

Hyp

ervi

sor

Hyp

ervi

sor


VETH

VNIC

27



Simpler Deployments ‒ Unifying the virtual and physical network

‒ Consistency in functionality, performance and management

Robustness ‒ Programmability of the infrastructure

‒ Troubleshooting, traffic engineering virtual and physical together

Performance ‒ Near bare metal I/O performance

‒ Improve jitter, latency, throughput and CPU utilization

Security ‒ Near bare metal I/O performance

‒ Improve jitter, latency, throughput and CPU utilization

Contrasting VM-FEX to Virtualised Switching Layers

28

VM-FEX General Details on UCS and Nexus 5500


UCS System Components Foundation

UCS 6100

UCS 6200

UCS 2208 IOM

UCS 2104 IOM

UCS Manager Embedded – Manages entire system

UCS Fabric Interconnect – UCS 6100 • 20x 10GE Ports – 1 RU • 40x 10GE Ports – 2 RU • Ethernet or FC Expansion Modules

UCS Fabric Interconnect – UCS 6200 • 48x Unified Ports (Eth/FC) – 1 RU • 32x base and 16x expansion

UCS Fabric Extender – UCS 2104 • 8x 10GE Downlinks to Servers • 4x 10GE Uplinks to FIs

UCS Fabric Extender – UCS 2208 • 32x 10GE Downlinks to Servers • 8x 10GE Uplinks to FIs

UCS Blade Server Chassis Flexible Bay Configuration Houses blades based on Industry-standard architecture

Adapters - M81KR VIC, M71KR, etc.

• Up to 2x 10GE ports

• M81KR: Up to 128 virtual interfaces

Adapter - UCS VIC 1280


• Up to 256 virtual interfaces

30


UCS System Architecture Foundation

2x 4 Link 80 Gbps per Chassis

2x 8 Links 160 Gbps per Chassis



Wire once for bandwidth, not connectivity Policy-driven bandwidth allocation Integrates as a single system into your data center

31


Nexus 5500 and 2200 Components for VM-FEX Foundation

Nexus 5548

Nexus 5596

Nexus 2232 FEX

Nexus 5548 • 48x Unified Ports (Eth/FC) max – 1 RU • 32x base and Universal GEM expansion for 16x • 1 expansion slot

Nexus 5596 • 96x Unified Ports (Eth/FC) max – 2 RU • 32x base and universal GEM expansion for 64x • 4 expansion slots

Nexus 2232 • 32x 10GE Downlinks to Servers • 8x 10GE Uplinks to Fis

Adapters – P81E


• Up to 128 virtual interfaces

32


UCS VM-FEX System View Deploying on a UCS B or C Series Infrastructure

3 4

1 3 4 5 6 7 8 Chassis IO Module A

1 2

Server Ports

3 4

1 3 4 5 6 7 8

1 2

Server Ports

VN Tag @ 10Gbe

2 2 Chassis IO Module B

Internal Connections

vfc0

2 3 4 5 6

0 1

vNIC1(s) vNIC2(s)

vfc1

2 3 4 5 6

ESX 4.0u1+ / RHEL KVM 6.1+ / MS Windows 8 Server

VM

-FEX

UCS 6x00 Physical Ports

Chassis IOM Ports


Chassis IOM Ports

VIC CPU

Virtual Interface Control Logic


vCenter Controlled interfaces on VMs

with forwarding rules enforced on dynamic

adapters and signaled on Private

Interfaces

HBA 0 vHBA0

HBA 1 vHBA1

veth10

1 1

8 7 2 1 Fiber Channel Uplink Ports


6 5 6 5

veth1 veth2 veth3 veth4 veth1 veth2 veth3 veth4 veth10 0

Mgmt Uplink

0

Mgmt Uplink

CIMC KVM etc.

Cisco Adapter

UCS B or C Series Server

UCS Fabric Interconnect B (port profiles) UCS Fabric Interconnect A (port profiles)

ESX Kernel Module / Libvirt / HyperV Extendable Switch

33


UCS VM-FEX System View Deploying on a UCS B or C Series Infrastructure

3 4

1 3 4 5 6 7 8 Chassis IO Module A

1 2

Server Ports

3 4

1 3 4 5 6 7 8

1 2

Server Ports

VN Tag @ 10Gbe

2 2 Chassis IO Module B

Service Console Kernel


vfc0

2 3 4 5 6

0 1

d-vNIC1 vNIC1(s) d-vNIC2 vNIC2(s) d-vNIC3

vfc1

2 3 4 5 6

VM

-FEX


Chassis IOM Ports


Chassis IOM Ports

VIC CPU






Interfaces

ESX Kernel Module / Libvirt / HyperV Extendable Switch

Attempts by Guest OS to improperly mark

traffic blocked

HBA 0 vHBA0

HBA 1 vHBA1 d-vNIC4

veth10

1 1



6 5 6 5


Mgmt Uplink

0

Mgmt Uplink

CIMC KVM etc.

UCS Fabric Interconnect B (port profiles) UCS Fabric Interconnect A (port profiles)

Cisco Adapter

UCS B or C Series Server

ESX 4.0u1+ / RHEL KVM 6.1+ / MS Windows 8 Server 34


UCS VM-FEX System View Deploying on a UCS C Series with Nexus 5500 Infrastructure

8

1 3 4 5 6 32 2232 FEX A

1 2

2232 Server Ports

8

1 3 4 5 6 32

1 2

2232 Server Ports

VN Tag @ 10Gbe

2 2 2232 FEX B


vfc0

2 3 4 5 6

0 1

vNIC1(s) vNIC2(s)

vfc1

2 3 4 5 6

ESX 4.0u1+

VM

-FEX

Nexus 55xx Physical Ports

2232 Fabric Ports


2232 Fabric Ports

VIC CPU






Interfaces

ESX Kernel Pass Through Module HBA 0 vHBA0

HBA 1 vHBA1

veth10

1 1




Mgmt Uplink

0

Mgmt Uplink

CIMC KVM etc.

Cisco P81E Adapter

UCS C Series Server

47 47

48 48

vPC Connections (veth’s not a vPC at FCS)

Nexus 55xx A (port profiles) Nexus 55xx B (port profiles)

35


UCS VM-FEX System View Deploying on a UCS C Series with Nexus 5500 Infrastructure

8

1 3 4 5 6 32 2232 FEX A

1 2

2232 Server Ports

8

1 3 4 5 6 32

1 2

2232 Server Ports

VN Tag @ 10Gbe

2 2 2232 FEX B


vfc0

2 3 4 5 6

0 1

vNIC1(s) vNIC2(s)

vfc1

2 3 4 5 6

VM

-FEX


2232 Fabric Ports


2232 Fabric Ports

VIC CPU






Interfaces

HBA 0 vHBA0

HBA 1 vHBA1

veth10

1 1




Mgmt Uplink

0

Mgmt Uplink

CIMC KVM etc.

Cisco P81E Adapter

UCS C Series Server

47 47

48 48

Service Console Kernel

d-vNIC1 d-vNIC2 d-vNIC3 d-vNIC4

6 5 6 5

Nexus 55xx A (port profiles) Nexus 55xx B (port profiles)

Attempts by Guest OS to improperly

mark traffic blocked

ESX 4.0u1+

ESX Kernel Pass Through Module

vPC Connections (veth’s not a vPC at FCS)

36


UCS General Baseline #1: Creating Dynamic vNICs

Policies are to automatically provision dynamics on Servers

Dependent on the number of Fabric Interconnect to IO Module connections ‒ (# IOM to FI links * 63) - 2

Setting a Dynamic Adapter Policy Up

37


UCS General Baseline #2: Building Service Profile

2 Statics – 1 to each UCS Fabric

Change dynamic vNIC connection policy to setup dynamics

Adding the Dynamic Policy and Static Adapters

38


UCS General Baseline #3: Building Port Profiles

Creating Port Profiles Includes: ‒ VLAN(s)

‒ Native and/or Tagging allowed

‒ QoS Weights and Flow Rates

‒ Upstream Ports to always use

Creating Folders of Network Access Attributes

39


UCS General Baseline #4: Building Port Profiles

Selecting High Performance will only Impact VMware deployment today

No problem if selected and used on other hypervisors

Enhanced Options like VMDirectPath with VM-FEX

40


UCS General Baseline #5: Communication with Manager

Same Plug-in Method used in Nexus 1000v

Tool discussed later to simplify the whole integration process

8 Separate managers today

Establishing Communication to Hypervisor Manager

41


UCS General Baseline #6: Publishing Port Profiles

Publish Port Profiles to Hypervisors and virtual switches within

4 Separate virtual switch per manager today

Exporting Port Profiles to these to Hypervisor Manager

42


UCS C2xx / N5500 General Baseline #1: Dynamic vNICs

Enable NIV (Network Interface Virtualization) on the P81E Adapter in the CIMC ‒ CIMC is the management Interface for the Cisco C2xx servers

‒ Choose the number of dynamics to configure (next slide)

Setting a Group of Dynamics on C2xx Servers

43


UCS C2xx / N5500 General Baseline #1: Dynamic vNICs

Enable vNICs then view the VM-FEXs Tab in the CIMC ‒ CIMC is the management Interface for the

Cisco C2xx servers

‒ UCS Standalone CIMC version 1.4 or greater required

‒ Minimum of 2 static vNICs defined

‒ Numbers of VM FEX’s (dynamic vNICs) are dependent on links from 5500 to 2232 if using FEX (Limit at 96 today)

‒ Nexus 5500 version 5.1(3)N1(1) or later required

Setting a Group of Statics and Dynamics on C2xx Servers

44


UCS C2xx / N5500 General Baseline #2: N5k Port Profiles nexus5500-1(config)# install feature-set virtualization

nexus5500-1(config)# feature-set virtualization

nexus5500-1(config)# feature vmfex

nexus5500-1(config)# feature npiv

nexus5500-1(config)# vethernet auto-create

nexus5500-1(config)# interface Ethernet100/1/27 << P81E Port

nexus5500-1(config-if)# switchport mode vntag

nexus5500-1(config)# port-profile type vethernet VM_VLAN_6

nexus5500-1(config-port-prof)# switchport access vlan 6

nexus5500-1(config-port-prof)# high-performance host-netio

nexus5500-1(config-port-prof)# port-binding dynamic

nexus5500-1(config-port-prof)# dvs-name all << To publish to the VM-FEX DVS

nexus5500-1(config-port-prof)# state enabled

Enabling VMFEX, and Creating Folders of Network Access Attributes

45


UCS C2xx / N5500 General Baseline #3: P81E Statics

Configuring a static profile for the fixed interfaces

One to each N5k in a pair

Select the vNIC, and the port profile to assign to it

These will be initially in the vSwitch in the out of box VMware configuration

Configuring Adapter FEX interfaces for Startup/Shutdown Operations

46


UCS C2xx / N5500 General Baseline #4: vCenter plugin

nexus5500-1(config)# feature http-server

Downloading and registration of plug-in per the other VM-FEX topologies

Enabling VMFEX, and Creating Folders of Network Access Attributes

47


UCS C2xx / N5500 General Baseline #5: Comm with Manager Establishing Communication to Hypervisor Manager

nexus5500-1(config)# svs connection vCenter1

nexus5500-1(config-svs-conn)# protocol vmware-vim

nexus5500-1(config-svs-conn)# extension-key UCSTME-Nexus5000-VMFEX-DVS

nexus5500-1(config-svs-conn)# remote ip address 172.25.177.227 port 80 vrf management

nexus5500-1(config-svs-conn)# dvs-name UCS_C2XX_VMFEX_DVS_1

nexus5500-1(config-svs-conn)# vmware dvs datacenter-name UCSTMELAB

nexus5500-1(config-svs-conn)# connect

Do this on Both Nexus 5500’s in the vPC Pair

nexus5500-1(config-svs-conn)# show svs connections

48

VM-FEX Implementation with VMware on UCS and Nexus 5500


VMware VM-FEX: Infrastructure Requirements

8 vCenters can be joined from UCS

Each vCenter can have 4 DVS’ within

Plug-In download and install method (unless Easy VM-FEX tool is used)

Hosts then use VUM Depot’s to install ESX module when bringing host into UCS DVS (unless Easy VM-FEX tool is used)

Enterprise+ required (as is for any DVS) on Host

Standard and above is required for vCenter

VMotion fully supported

VMDirectPath with VM-FEX is possible (Hypervisor Bypass)

‒ Cisco UCS with VM-FEX linked to in vSphere 5 networking guide – page 42

‒ vCenter VM->Properties->Resources – need to reserve all guest memory

Versions, Licenses, etc.

50


VMware VM-FEX: vCenter View

Uplinks from ESX hosts shown on right ‒ These are the statics for overhead

VM vNICs shown in port groups on left ‒ Port Groups are from Port Profiles

sent in from UCSM or Nexus 5500

Normal view of VM vNICs, MAC, Port numbers, etc.

View from the Administrator

51



Match the Memory Reservation with the Limit

Fundamental requirement of all DirectPath deployments

Only Supported Guests will get DP with VM-FEX ‒ Windows Server 2008 SP2, Windows Server 2008 R2, RHEL 6.x, SLES11 SP1

View from the VM Settings to Get DirectPath with VM-FEX

52



Simply add vNICs

Define normal VMXNET3 type

Select Port Group to put adapter into

Displays if the DirectPath with VM-FEX is active

Other Adapters can remain in emulated mode if desired

View from the VM Settings

53


VMware VM-FEX: vCenter View View from the vCenter

54


VMware VM-FEX: Administrator View View from the UCSM

55


VMware VM-FEX: Administrator View

User can see where the VM’s map to the Veth ports on the 5500’s

Normal NX-OS command set to view/manage/SPAN etc as in physical ports

View from the Nexus 5500

56

VM-FEX Implementation with KVM on UCS


RHEL KVM VM-FEX: Infrastructure Requirements

VM-FEX is available for KVM on only UCS Managed Deployments Today

Install Red Hat as Virtualization Host

Unlike VMware no VEM to load (utilizes libvirt)

Works with live migration feature for VMs

Scripted nature of configuration at FCS ‒ No current RHEV-M for RHEL KVM 6.x

‒ RHEV-M 3.0 will have RHEL 6.2 hooks for VM-FEX configuration assistance

MacVTap has 3 distinct modes ‒ Bridge mode for normal end points connecting together within host (Adapter FEX has value here)

‒ 802.1Qbg “VEPA” for traffic hair-pinning on next upstream device

‒ 802.1Qbh “private-mode” for traffic always passing to controlling bridge (UCS FI)

VM-FEX uses private-mode

Virtual Machine interface management via editing of VM domain XML file


58


RHEL KVM VM-FEX: Virt-manager View

RHEL virt-manager to do simple VM operations

Can start, stop, open, migrate VM’s with VM-FEX connections also

RHEV-M will be able to present VM-FEX port profiles natively with 3.x


59


RHEL KVM VM-FEX: CLI View

Virsh set of commands to control VM’s

Create, Start, Stop, Migrate, etc. of VM’s with VM-FEX included


60


RHEL KVM VM-FEX: VM vNIC View

Adapters can be created in virt-manager wizard and MAC assigned

Edit the domain.xml file to make the adapters VM-FEX

Bring in the port profile here

Then VM will operate as normal

No reliance on bridge domains


61


VMware VM-FEX: Administrator View

Same port profiles can be used to KVM VM’s

Live Migration fully supported

View from the UCSM

62

VM-FEX Implementation with Hyper-V on UCS


HyperV VM-FEX: Infrastructure Requirements

VM-FEX is available for HyperV on only UCS Managed Deployments at shipment of Windows 8

HyperV Role Enabled on Windows 8 Servers

For Live Migration, MS Cluster built with shared storage

‒ VM-FEX with Live Migration fully supported

HyperV Networks defined as shown here

‒ Through HyperV Manager GUI

‒ Through PowerShell Applets

Cisco Extension to the HyperV Extensible Switch infrastructure

Systems Center Virtual Machine Manager (SCVMM) 2012 version with Windows 8 Server support will be needed for manager integration

‒ Fully supported via PowerShell until the SCVMM ships


64


HyperV VM-FEX: VM vNIC View

Install the VIC device

Install the Cisco VmFex Forwarding Extension

‒ .\vmfextool.exe -i “E:\VmFex\ReleaseDrivers\cscovmfext\cscovmfext.inf" "Csco_FEXT"

Create a IOV Switch

‒ New-VMSwitch -Name "Cisco-Network" -NetAdapter $externalnic -SwitchType External -AllowManagementOS 1 -Notes "Cisco Private Network Traffic Switch" -EnableIov 1

‒ Enable-VMSwitchExtension -VMSwitchExtensionName "Cisco VmFex Forwarding Extension

Insert the Cluster UUID and Name to match the UCSM

‒ .\Cisco_SwitchProp-Add.ps1 "Cisco-Network" "38c79463-e5d3-46f6-ba47-b35719f15c70" "Msft-clus"

Add an SR-IOV adapter to the VM with the command

‒ .\PortProfileSetting-Add.ps1 "VMName" "PortProfileId" "PortProfileName" "NicID"

Key PowerShell Components

65


HyperV VM-FEX: Failover Cluster Manager View View from the Administrator

66



Virtual Network Manager to configure the switch

PS scripts to configure the Cisco extension to the HyperV switch


67



VM is created first

PS scripts add the network policy from our port-profiles behind Adapters


68


VMware VM-FEX: Administrator View View from the UCSM

69

Easy-VMFEX tool usage


Easy VM-FEX Tool

VMware solution only today with UCS

Quick System Bringups

Assumption of 1 management interface per ESX host

‒ Optional vMotion / FT logging also handled

All supported versions of VMware that VM-FEX supports

‒ Enterprise+ or Evaluation

Can define some defaults in text file

vCenter folders OK

Server needs Dynamic vNICs on Service Profile (will check)

Deployment name limited to 8 characters in tool

UCSM respository for ESX kernel model, or separate tool to pull from VMware online to a dedicated directory locally

Tool Usage

71


Easy VM-FEX Tool Tool View

72


VM-FEX Performance Review

Virtual Switch, CPU at ~65%

VM-FEX, CPU at ~ 37%

Test of 10 VM’s running HTTP and FTP Gets with IxLoad

73


VM-FEX Performance Review

3 IxVM’s, sending fixed 3.33G flows to a 4th VM All on same ESX host to maximize the differences in operations

‒ Virtual switch with VMs on same host – flows contained within ESXi 5 host ‒ VM-FEX with VMs on same host – flows traverse the chassis IOM/FI, or N2k/5k ‒ Higher Bandwidth AND Lower Latency between VM’s

Test of 4 IxVM’s – Virtual Switching CPU @ 88%, VM-FEX CPU @ 52%

74

Summary


Summary

FEX technologies can reduce managed device count

FEX technologies will greatly reduce cabling overhead

VM-FEX is terminating these virtual links directly on the VMs

Closely maps to the physical server model for operations and management

Multiple Hypervisors are supported with advanced features

Bandwidth can be engineered identically to physical infrastructures today

Latency can surpass local virtualized switching by moving away from virtual switching store and forward buffering, “tree’s” of ASIC traversals, to a uniform port controller and switch fabric model

76


BRKCOM-2005 Recommended Viewing

www.YouTube.com/ciscodatacenter Playlist UCS Technical Videos http://www.youtube.com/ciscodatacenter#p/c/F04A2C6AA04DF055

Overview Cisco UCS Advantage http://www.youtube.com/watch?v=IW4zHXIjpPU

UCS Advantage Videos on YouTube

77


Category Title URL UCS server Service Profiles and Templates http://www.youtube.com/watch?v=JW-YtVN75R0

UCS server Organizations and Roles http://www.youtube.com/watch?v=tb-L0zv3If

UCS server Extended Memory Technology http://www.youtube.com/watch?v=kS3ehPRcVDo

UCS server Server Pre-Provisioning http://www.youtube.com/watch?v=o7BuEE3hNPE

UCS server BIOS Policies http://www.youtube.com/watch?v=Pr6EptC9JXQ

UCS server RAID Policies http://www.youtube.com/watch?v=Vcs56wjUWuI

UCS server Firmware Policies http://www.youtube.com/watch?v=vjj8Xz0NqI4

UCS server Server Pools and Qualification Policies http://www.youtube.com/watch?v=KTw7M3T-VOw

UCS server Maintenance Policies http://www.youtube.com/watch?v=QQTlm98NgTI

UCS server High Availability During Upgrades http://www.youtube.com/watch?v=57HXMGn88HA

UCS server Monitoring with BMC BPPM http://www.youtube.com/watch?v=mdoEZf7tM5E

UCS server Microsoft Hyper-V on UCS http://www.youtube.com/watch?v=G3x_YOYK-Fo


78



Category Title URL

UCS I/O Adapter Templates http://www.youtube.com/watch?v=KpVEn3DhfOM

UCS I/O Network Interface Virtualization http://www.youtube.com/watch?v=njjbCEblxVc

UCS I/O Adapter Fabric Failover http://www.youtube.com/watch?v=tlu8RSq6T_M

UCS I/O Extend the Network to the Virtual Machine http://www.youtube.com/watch?v=Ylizxq18yxE

UCS I/O Traffic Analysis of All Servers http://www.youtube.com/watch?v=PHTdXy_8Zdg

UCS I/O Ethernet Switching Modes http://www.youtube.com/watch?v=roX8MRN66UM

UCS I/O Fibre Channel and Switch Modes http://www.youtube.com/watch?v=VSetsgOYYCo

UCS I/O FC Port Channels and Trunking http://www.youtube.com/watch?v=PpzKPguRTXc

79


Category Title URL

UCS Infrastructure

Lights-Out Management http://www.youtube.com/watch?v=QEO1d_1vTxs

UCS Infrastructure

Easy VM-FEX Deployment http://www.youtube.com/watch?v=0aAuj80cNvg

UCS Infrastructure

Server Power Grouping http://www.youtube.com/watch?v=EgoFe33YoD8

UCS Infrastructure

Blade and Rack-Mount Management http://www.youtube.com/watch?v=aOsx4YMiOho

UCS Infrastructure

Manager Platform Emulator http://www.youtube.com/watch?v=ZNNrs2e0wvk

UCS Infrastructure

Cisco Developer Network and Sandbox http://www.youtube.com/watch?v=Syhl6SAiwew


80


Complete Your Online Session Evaluation Give us your feedback and you

could win fabulous prizes. Winners announced daily.

Receive 20 Passport points for each session evaluation you complete.

Complete your session evaluation online now (open a browser through our wireless network to access our portal) or visit one of the Internet stations throughout the Convention Center.

Don’t forget to activate your Cisco Live Virtual account for access to all session material, communities, and on-demand and live activities throughout the year. Activate your account at the Cisco booth in the World of Solutions or visit www.ciscolive.com.

81


Final Thoughts

Get hands-on experience with the Walk-in Labs located in World of Solutions, booth 1042

Come see demos of many key solutions and products in the main Cisco booth 2924

Visit www.ciscoLive365.com after the event for updated PDFs, on-demand session videos, networking, and more!

Follow Cisco Live! using social media: ‒ Facebook: https://www.facebook.com/ciscoliveus

‒ Twitter: https://twitter.com/#!/CiscoLive

‒ LinkedIn Group: http://linkd.in/CiscoLI

82

virtual machine fabric extender (vm-fex)...

Documents