端到端数据中心虚拟化 - cisco

Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. 1

端到端数据中心虚拟化

数据中心解决方案部

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 2

日程

• Data Center Virtualization Overview

• Front-End Data Center Virtualization

Core Layer

Aggregation Layer

Networking Services

Access Layer

• Server Virtualization

Hypervisors

Virtual Access Layer

Virtualized Services

Server IO Virtualization

• Back-End Virtualization

Virtual HBA & NPV

Unified IO & FCoE

SAN & Storage

• Q&A


影响IT和数据中心的关键趋势

Server Virtualization — higher performance

LAN and Storage convergence

VM-Level awareness

Workload provisioning

Applications availability

Drive for Green—power, cooling and space

The need to reduce costs and/or maximize profits

IT as business enabler


Virtualization Touches Half (at Least …)

Server Virtualization — higher performance

LAN and Storage convergence

VM-Level awareness

Workload provisioning

Applications availability

Drive for Green—power, cooling and space

The need to reduce costs and/or maximize profits

IT as business enabler


• The Application Services provided by the Network need to respond and be aligned to meet the new geometry of the VMs

• Close interaction required between the assets provisioning Virtualized infrastructure and the Application Services supporting the Virtual Machines.

虚拟化数据中心方法论


Moving to a fully virtualized Data Center, with Any to Any Connectivity

迈向统一网络

• Fully unified I/O delivers the following characteristics:

Ultra High Capacity 10Gbps+

Low latency

Loss Free (FCoE)

• True ―Any to Any‖ Connectivity is possible as all devices are connected to all other devices.


Access L

ayer

Ag

gre

gati

on

Layer

Co

re L

ayer

SA

N E

dg

e

SA

N C

ore

虚拟化数据中心架构(3 层)


Data Center Row 1

数据中心架构演化对传统网络设计的挑战

Data Center Row 2

Hypervisor based server virtualization and the associated capabilities (vMotion, Live Migration, etc.) are changing multiple aspects of the Data Center design

Where is the server now?

Where is the access port?

Where does the VLAN exist?

Any VLAN Anywhere?

How large do we need to scale Layer 2?

What are the capacity planning requirements for flexible workloads?

Where are the policy boundaries with flexible workload (Security, QoS, WAN acceleration, …)?


虚拟化网络架构演进和考虑点

Typical DC Challenges

L2 Fate-sharing

VLAN Location

L2 Adjacency

Higher Scale

L3 Access

App Environments

What are the implications…

Dynamic ―routing protocol‖ for L2 (e.g.: IS-IS)

Any VLAN anywhere resonates well

Lower oversubscription

Larger subnet sizes

Global VLANs

Specific app environments Designs

Access Ports in

management domain

Density and

Capabilities of

access switch

Density & quantity of

aggregation

switches

Classic Pod Modern Pod


VM #4

VM #3

VM #2

Host Facing

vPC

Nexus 1000v & vPATH

VSG, vWAAS

Virtualized Interfaces Adapter FEX, VM-FEX

& FCoE

VDC: Virtual Device

Contexts

NX-OS – Modular Operating System common

across the DC

ISSU – True non-stop operations

vPC – Between Nexus layers for bi-sectional

bandwidht use (no STP loops)

DCNM – Consolidated

Configuration and Management

Unified Fabric: Multi-Hop FCoE

Unified Ports

FCoE

FC

FEX Architecture

FET + FEX: Cabling cost efficiencies


Leaf

Layer

Sp

ine L

ayer

Converged FCoE link

Dedicated FCoE link

FC

1 / 10GE

DR Data Center

OTV: Layer 2 Extension

ASA 5500

ACE

ACE


• FabricPath enabled for LAN traffic

• Dual Switch core for SAN A & SAN B

• All Access and Aggregation switches are FCoE FCF switches

• Dedicated links between switches are VE_Ports

• Storage VDC (Nexus 7000 only) for additional operation separation at high function agg/core (aka spine)

Improved HA and scale over vPC (ISIS, RPF, … and N+1 redundancy)

SAN can utilize higher performance, higher density, lower cost Ethernet switches (including unified ports)

(*) FC connectivity to storage only available on Nexus 5000/5500. FCoE target and NAS / iSCSI target connectivity to any Nexus switch.

用二层多路径（FabricPath)来支持双网SAN架构

L2

L3

CNA FC (*) FCoE

Fabric ‗A‘

Fabric ‗B‘

FCF

FCF

FCF

FCF

VE

Converged FCoE link

Dedicated FCoE link

FC

1 / 10GE

FabricPath


NAS iSCSI

Leaf

Layer

Sp

ine L

ayer


日程



Core Layer

Aggregation Layer

Networking Services

Access Layer


Hypervisors





Virtual HBA & NPV

Unified IO & FCoE

SAN & Storage

• Q&A


前端：核心层

13


虚拟交换机@ Nexus 7000

Kernel

Infrastructure

Protocol Stack

VDCA

Nexus 7000 Physical Switch

VDC A

Pro

ce

ss A

BC

Pro

ce

ss D

EF

Pro

ce

ss X

YZ

…

Protocol Stack

VDCB

VDC B

Pro

cess A

BC

Pro

cess D

EF

Pro

ce

ss X

YZ

…

Process ―DEF‖ in VDC B

Crashes

Process DEF in VDC A Is

Not Affected and Will

Continue to Run

Unimpeded

A

B

C

D

A B

C

D


FIB TCAM

Size 128K

ACL TCAM

Size 64K

FIB TCAM

Size 128K

FIB TCAM

Size 128K FIB TCAM

Size 128K

VDC-1

IP routes: 20K

ACL entries: 10K

VDC-2 IP routes: 100K

ACL entries: 50K

ACL TCAM

Size 64K

VDC-3 IP routes: 100K

ACL entries: 50K

ACL TCAM

Size 64K

ACL TCAM

Size 64K

Linecard 1 Linecard 2

Linecard 3 Linecard 4

1 : N 虚拟交换机隔离的资源分配域(3层)


前端: 汇聚层

Gigabit Ethernet

10 Gigabit Ethernet

10 Gigabit DCB

4/8Gb Fiber Channel

10 Gigabit FCoE/DCB

16


Catalyst 6500 虚拟交换系统(VSS）转发运作

Virtual Switch Domain

Switch 1—Control Plane Active Switch 2—Control Plane Hot Standby

Virtual Switch Domain

Switch 1—Data Plane Active Switch 2—Data Plane Active


Virtual Port Channel (vPC) 双活2层连接

vPC is a Port-channeling concept extending link aggregation to two separate physical switches

Allows the creation of resilient L2 topologies based on Link Aggregation.

Eliminates the need for STP in the access-distribution Layer

Enable seamless VM Mobility, Server HA Clusters

Scale Available Layer 2 Bandwidth

Dual-homed server operate in active-active mode

Simplify Network Design

Available on Nexus 7000 and Nexus 5000 / 5500

Bi-sectional BW with vPC

L2

SiSi

Non-vPC vPC

SiSi

Virtual Port Channel

Physical Topology Logical Topology


• Multilayer vPC can join eight active member ports of the port-channels in a unique 16-way port-channel*

• vPC peer load-balancing is LOCAL to the peer device

• Each vPC peer has only eight active links, but the pair has 16 active load balanced links (M-series LC)

• F-series Nexus 7000 line cards support 16 way active port-channel load balancing, providing for a 32 way vPC port channel

Nexus 7000

Nexus 5000

32-way port channel

双vPC 域实现与设计 32-Way Port-Channel – Double-sided VPC

Double-sided vPC architecture


Router/Firewall

7k1

Po1

7k2 P P

P

Routing Protocol Peer

Dynamic Peering

Relationship

P

Layer 3 and vPC 设计 Router/Firewall on a stick with VDC

Switch

Po2

vPC Domain

Physical Device

Layer 3 VDC

Router/Firewall

7k1

Po1

7k2

P P

P

Switch

Po2

7k3 7k4

Layer 2 vPC

Layer 2 VDC

no Dynamic Routing

on vPC VLANs


思科 FabricPath 扩展简化二层以太网

-All Links Active

Traditional Spanning Tree Based Network

Up to 16 Agg

switches

-Blocked Links

Cisco FabricPath Network

160+ Tbps

switching capacity

Eliminate Spanning tree limitations

Multi-pathing across all links, high cross-sectional bandwidth

High resiliency, faster network re-convergence

Any VLAN, any where in the fabric eliminate VLAN Scoping


Spanning-Tree vPC FabricPath

Pod

Bandwidth

Active Paths

Up to 10 Tbps Up to 20 Tbps Up to 160 Tbps

Single Dual 16 Way

Infrastructure Virtualization and Capacity

Layer 2 Scalability

Up to 16 Switches

思科Nexus 架构的灵活性


前端:网络服务

Gigabit Ethernet

10 Gigabit Ethernet

10 Gigabit DCB

4/8Gb Fiber Channel

10 Gigabit FCoE/DCB

Networking

Services

23


数据中心虚拟化服务

v5

v105

v6 v7

v107

v2081

v2082

v2083

...

v206 v207

v206

BU-4 BU-2 BU-3

v105

v108

BU-1

1

2

3

4

* vX = VLAN X

**BU = Business Unit

VRF

VRF

VRF VRF VRF

v208

―Front-End‖ VRFs (MSFC)

Firewall Module Contexts

ACE Module Contexts

―Back-End‖ VRFs (MSFC)

Server Side VLANs

v207

3

4

v8


VDC 1

VDC 2

Cat6500

VSS

Sub-Agg

VDC

Services

Contexts

Agg

VDC

Services are ―Sandwiched‖ between Nexus VDCs

Stateful Firewall: Virtual Contexts, Transparent mode

ACE Load-balancer: Routed Two-arm mode, Virtual Contexts

三明治式的虚拟服务设计

Merging access/aggregation without sacrificing the functional management of each layer

Inter Tenant (VM-to-VM and Multi-tier Flows), policy Management (Security, QoS, BW etc)

Operational isolation (change mgmt, span of control) of access-layer versus core/aggregation

Rationale for VDC sandwich design


虚拟服务机箱设计

Reduced complexity with multi-tenant design.

Network Multi-tenancy definition and scope will not be limited to service blade.

Container definition is not tied just to services blade

Improved Convergence and Scalability

Service will be isolated via L3 port-channel to/from VSS.

Isolation and flexibility on insertion of appliance based model.

Reduced ―always inline‖ effect between VSS and Aggregation-layer

Better technology and feature integration

Ease of Multicast support

Separation of core VDC – freeing VDC resources at the aggregation layer for Storage & OTV

Aggregation VDC will not be split (agg / sub-agg) and will represent single L2/L3 boundary for all compute/storage flows

Services

VSS

Aggregation

Core

L3


在FC刀片交换机上采用NPV

• Eliminates edge FC switch Domain ID

• Edge FC switch acts as an NPIV host

• Simplifies server and SAN management and operations

• Increases fabric scalability


Overlay Transport Virtualization (OTV)

• Ethernet traffic between sites is encapsulated in IP: ―MAC in IP‖

• Dynamic encapsulation based on MAC routing table

• No Pseudo-Wire or Tunnel state maintained

OTV at a Glance

Communication between MAC1 (site 1) and MAC2 (site 2) Server 1

MAC 1

Server 2

MAC 2

OTV OTV

MAC IF

MAC1 Eth1

MAC2 IP B

MAC3 IP B

IP A IP B

Encap Decap

MAC1 MAC2 IP A IP B MAC1 MAC2 MAC1 MAC2


Eth 4

Eth 3

MAC TABLE

VLAN MAC IF

100 MAC 1 Eth 2

100 MAC 2 Eth 1

100 MAC 3 IP B

100 MAC 4 IP B

MAC 2

MAC 1

OTV 数据平面: 单播

Core

MAC 4

MAC 3

OTV

External

IP A

External

IP B

West East

L2 L3 L3 L2

OTV Inter-Site Traffic

MAC Table contains

MAC addresses reachable through

IP addresses

OTV

Encap 2

Layer 2

Lookup

1

No Pseudo-Wire state is maintained.

The encapsulation is done based on a Layer 2 destination lookup.

3 Decap 4 MAC 1 MAC 3

6

MAC TABLE

VLAN MAC IF

100 MAC 1 IP A

100 MAC 2 IP A

100 MAC 3 Eth 3

100 MAC 4 Eth 4

Eth 1

Eth 2

Layer 2

Lookup

5

MAC 1 MAC 3

IP A IP B MAC 1 MAC 3 MAC 1 MAC 3 IP A IP B MAC 1 MAC 3


跨POD网络扩展

Cisco Innovation towards an end-end Fabric:

• Cisco FabricPath: Scalable Fabric for Application Deployment Flexibility

• OTV : Layer 2 extensions over Layer 3 for distributed Clustered Applications

• LISP: IP mobility, optimized routing

Data Center Interconnect Extension

Overlay Transport Virtualization (OTV)


前端: 接入层

Gigabit Ethernet

10 Gigabit Ethernet

10 Gigabit DCB

4/8Gb Fiber Channel

10 Gigabit FCoE/DCB

31


架顶@ 1/10GE/FCoE: Nexus 2200 (网络扩展器— FEX)

Nexus

2200


Nexus

2000

1GE Rack

Mount Servers

Nexus

2000

10GE Rack

Mount Servers

Nexus

4000

10GE Blade

Switch w/ FCoE

(IBM/Dell)

Cisco

UCS Nexus

2000

1 & 10GE

Blade Servers

w/ Pass-Thru

10GE Rack

Mount Servers

Direct Attach

10GE

Nexus 7000

Unified Access Layer

Nexus 5000

UCS Compute

Blade & Rack

Cisco Nexus 2000 Unified Server Access Architecture • N2K inherits the features from parent switch


To2R: Nexus 2200 部署举例

Rack 1 Rack 2

Access Layer

Rack 1 Rack 2Rack 1 Rack 2

Aggregation LayerNexus 7000 Nexus 7000

Nexus 5500 Nexus 5500

Nexus 2200 Nexus 2200 x4 x4x4x4

x4 x4x4x4

Rack 1 Rack 2Rack 1 Rack 2 Rack 12 Rack 1 Rack 2 Rack 12

vPC

vPC


虚拟接入交换机POD

• Cisco Nexus 5x00 and 2200 represent a virtual access switch POD

• Nexus 7000 at Aggregation Layer

Nexus 5x00/2200 Virtualized Access

Switch PODs . . .

NO Loop

VPC pair

NO STP


S10 S20 S30 S40

S100 S101 S200 FabricPath

Server

L1 L2 L4 L3

L5 L6 L7 L8

L9 L10 L11 L12

采用FabricPath的逻辑图: 无二层环路的分布式拓扑

Unified Computing System (UCS)

Virtual Access Switch POD

(Nexus 7000 / 5x00 + Nexus 2200)

Virtual Blade Switching (VBS)


日程



Core Layer

Aggregation Layer

Networking Services

Access Layer


Hypervisors





Virtual HBA & NPV

Unified IO & FCoE

SAN & Storage

• Q&A


前端: 服务器层

Gigabit Ethernet

10 Gigabit Ethernet

10 Gigabit DCB

4/8Gb Fiber Channel

10 Gigabit FCoE/DCB

Cisco UCS Servers Layer

38


SAN B

什么是思科UCS?

• UCS = Unified Computing System

• Single, scalable integrated system

Mgmt SAN A LAN

Network + compute Virtualization

Dynamic resource provisioning


网络对服务器完全可见

•UCS Service Profiles Capture more than MAC & WWN

MAC, WWN, Boot Order, Firmware, network & storage policy

•Stateless compute where network & storage see all movement

Better diagnostics and QoS from network to blade, policy follows

SAN

LAN Chassis-1/Blade-5

Chassis-9/Blade-2

Server Name: SP-A

UUID: 56 4d cd 3f 59 5b 61…

MAC : 08:00:69:02:01:FC

WWN: 5080020000075740

Boot Order: SAN, LAN

Service Profiles deliver Service Agility

regardless of Physical or Virtual Machine


Total Server Deployment

18 Servers

以Service Profiles构建弹性数据中心

Today‘s Deployment:

Provisioned for peak capacity

Spare node per workload

Workload Server Capacity Needed Server HW HA Total Servers

Oct Nov Dec Jan

Web Servers 5 7 6 5 1 hot spare 8

Oracle RAC 3 3 3 4 1 hot spare 5

VMware 3 3 4 4 1 hot spare 5

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Web Servers

Oracle RAC VMware


Total Server Deployment

14 Servers

Reduction of 4 Servers

22% CapEx Savings

无状态计算@ UCS

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Web Servers

Oracle RAC VMware

Old Deployment:

Blade

Blade

HA Spare Burst Capacity

Cisco’s Deployment:

• Resources provisioned based

on business need

• Still HA with fewer spares

Cisco UCS Deployment: (still 18 Service Profiles)

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Web Servers

Oracle RAC VMware


What Happens When We Mix Network and Server Virtualization ?

43


服务器虚拟化时网络面临的问题

Problems:

• Dynamic Migration of VMs may move them across physical server ports—policy must follow

• Impossible to view or apply policy to locally switched traffic

• Need collaboration between network and Virtualization admin

VLAN 101


Gigabit Ethernet

10 Gigabit Ethernet

10 Gigabit DCB

4/8Gb Fiber Channel

10 Gigabit FCoE/DCB

Virtual Access Layer Cisco Nexus 1000v

虚拟接入层 @ 虚拟化服务器

45


Nexus 1000V VSM

Cisco Nexus 1000V 架构

Nexus 1000V VSM

vCenter

Virtual Supervisor Module (VSM)

Virtual or Physical appliance running Cisco NXOS (supports HA)

Performs management, monitoring, & configuration

Tight integration with VMware vCenter

Virtual Ethernet Module (VEM)

Enables advanced networking capability on the hypervisor

Provides each VM with dedicated ―switch port‖

Collection of VEMs = 1 vNetwork distributed Switch

Cisco Nexus 1000V Installation

ESX & ESXi

VUM & Manual Installation

VEM is installed/upgraded like an ESX patch

vSphere

Nexus

1000V

VEM

vSphere vSphere

Nexus 1000V

VEM

Nexus

1000V

VEM

VM VM VM VM VM VM VM VM VM VM VM VM

Physical Server Physical Server Physical Server


单一的控制和管理平面

Even if the Nexus 1000V is a distributed switch. It looks like a single switch from control plane and management plane perspective

Protocol like CDP, Netflow, SNMP are

manage from one location the VSM

(Virtual Supervisor Module)

A B C


Nexus 1010: ―Virtual Service Blade‖ Manager

Nexus 1010 Manager: Cisco management experience

Manages virtual service blades

Nexus 1000V VSM Nexus 1000V VSM Nexus 1000V VSM Nexus 1000V VSM

Nexus 1010 Manager

Network Analysis

Module*

* Optional virtual service blade add-on


分布式数据平面

The Virtual Ethernet Module (VEM) is in the Data path

The Virtual Supervisor Module is only doing control plane and management function

Each Virtual Ethernet Module forwards packets independent of each other

A B C D E F A B C


Virtualized Services (VSN)

Nexus 1000v

虚拟服务节点 @ 虚拟化服务器

Virtual Service Gateway

vWAAS

Gigabit Ethernet

10 Gigabit Ethernet

10 Gigabit DCB

4/8Gb Fiber Channel

10 Gigabit FCoE/DCB

50


Hypervisor

Traditional Service Nodes

Virtual Contexts

VSN部署模式 (Virtual Service Nodes)

VLANs

Hypervisor

Redirect VM traffic via VLANs to external (physical) appliances 1

App Server

Database Server

Web Server

Apply hypervisor-based network services 2

App Server

Database Server

Web Server

VSN

Virtual Service Nodes

VSN


VDC vApp

Tenant 1

VDC vApp

Tenant 2

Nexus 1000V

VMWare Hypervisor

UCS (physical web servers)

Single VNMC Manager


Nexus 1000v的vPATH 截取

vPATH Interception is configured on Server VM‘s Port Profile in both directions to redirect to a VSN

Server traffic is intercepted by vPATH interception in VEM and redirected to a VSN

VSN egress traffic forwarded without further vPATH interception.

Upstream

Switch

VSM

VSN Server

VM

VEM

vPATH

Interception

In/Out


Cisco 虚拟安全网关(VSG) 以vPATH智能导引流量

Nexus 1000V

Distributed Virtual Switch

VM VM VM

VM VM

VM

VM VM VM

VM

VM

VM VM VM

VM VM VM VM

VM

vPATH

VNMC

Log/Audit

Initial Packet Flow

VSG

1


Nexus 1000V


VM VM VM

VM VM

VM

VM VM VM

VM

VM

VM VM VM

VM VM VM VM

VM

vPATH

VNMC

Log/Audit

Initial Packet

Flow

Flow

Access Control

VSG

1

2



Nexus 1000V


VM VM VM

VM VM

VM

VM VM VM

VM

VM

VM VM VM

VM VM VM VM

VM

vPATH

VNMC

Log/Audit

Decision

Caching

VSG 3

2 Initial Packet

Flow 1

Flow

Access Control



Nexus 1000V


VM VM VM

VM VM

VM

VM VM VM

VM

VM

VM VM VM

VM VM VM VM

VM

vPATH

VNMC

Log/Audit

VSG

4

2 Initial Packet

Flow 1

Decision

Caching 3

Flow

Access Control



Nexus 1000V


VM VM VM

VM VM

VM

VM VM VM

VM

VM

VM VM VM

VM VM VM VM

VM

vPATH

Remaining packets

from flow

ACL offloaded to

Nexus 1000V

VNMC

Log/Audit

VSG



Virtual Network

Management Center

(VNMC)

VM context aware rules Context aware

Security

Establish zones of trust Zone based Controls

Policies follow vMotion Dynamic, Agile

Efficient, Fast, Scale-out Software Best-in-class

Architecture

Security team manages security Non-disruptive

Operations

Central mgmt, scalable deployment, multi-tenancy Policy Based

Administration

Virtual Security

Gateway (VSG)

XML API, security profiles Designed for

Automation

Cisco 虚拟安全网关(VSG) 更优的安全解决方案


vWAAS vPATH截取

• Interception based on port-profile policy configured in Nexus 1000v

• Bidirectional Interception - (no IN/OUT configuration)

• Pass-through traffic automatic bypass

vCenter Server Nexus 1000v VSM

Cisco UCS x86 Server

Web

Server 1 App

Server

VMware ESXi Server

Nexus 1000V vPATH

vWAAS


Cisco UCS

Nexus 5000 & Nexus 2000 Top-of-Rack

FIP

FIP

Nexus 5500 & Nexus 4000


Nexus 5500 & Nexus 2200 Top-of-Rack

服务器 IO 虚拟化

Gigabit Ethernet

10 Gigabit Ethernet

10 Gigabit DCB

4/8Gb Fiber Channel

10 Gigabit FCoE/DCB

61


*IEEE 802.1Qbh Pre-Standard

网络扩展器演进分布式模块化系统

FEX Architecture

Consolidates network management

FEX managed as line card of parent switch

Uses Pre-standard IEEE 802.1Qbh

IEEE 802.1 Qbh*

Many applications

require

multiple interfaces

One Network Parent Switch to Top of Rack

Legacy

FEX

Network

Administrator


Legacy

IEEE 802.1 Qbh* Adapter FEX

Consolidates multiple 1Gb interface into a single 10Gb interface

Extends network into server


One Network Parent Switch to Adapter

IEEE 802.1 Qbh*

Adapter FEX

Many applications

require

multiple interfaces

FEX


Network

Administrator



Legacy

IEEE 802.1 Qbh*

Adapter FEX

Hypervisor

One Network Virtual Same As Physical

VM-FEX

Consolidates virtual and physical network

Each VM gets a dedicated port on switch


IEEE 802.1 Qbh* IEEE 802.1 Qbh*

VM network

managed by

Server

administrator

VM-FEX

FEX

Network

Administrator




Hypervisor

IEEE 802.1 Qbh*

One Network Parent Switch to Application

Single Point of Management

FEX Architecture

Consolidates network management

FEX managed as line card of parent switch

Adapter FEX

Consolidates multiple 1Gb interface into a single 10Gb interface

Extends network into server

VM-FEX

Consolidates virtual and physical network

Each VM gets a dedicated port on switch

IEEE 802.1 Qbh* IEEE 802.1 Qbh*

Adapter FEX Legacy

Manage network all

the way to

the OS interface –

Physical and

Virtual

FEX

VM FEX


Network

Administrator


Cisco UCS VIC 简介 Mezzanine Card for B-Series and C-Series

PCIe x16

10GbE / FCoE

User Definable

vNICs

Eth

0

FC

1 2

FC

3

Eth

58

Converged Network Adapter (CNA) designed for both single-OS and VM-based deployments

• Virtualized in Hardware

• PCIe compliant

High Performance

• 2x 10Gb

• 500K+ IOPS

The OS/Hypervisor sees up to ~58 distinct PCIe devices

• Ethernet vNIC and FC vHBA

• Management from the network

VM-FEX (aka VN-Link in Hardware): Ideal for Virtualization Environments

• Bypass vSwitch to deliver VN-Link in hardware

• Tight integration with Vmware vCenter


VN-Link功能小结 Cisco VIC + UCS Cisco VIC + UCS

with VMDirectPath

Generic Adapter

+ Nexus 1000V

Generic

adapter Cisco

VIC as

Adapter

FEX

Cisco

VIC as

VM-FEX

Cisco VIC + Nexus 1000V

(suggested deployment)

GREATER FLEXIBILITY/SCALABILITY, RICH

FEATURE SET AND FASTER TIME TO

MARKET

HIGHER PERFORMANCES & BETTER I/O

MANAGEMENT

VIC +

UCS 6100

VMDirectPath

Generic Adapter &

Nexus 1000v

VIC & Nexus 1000v

VIC +

UCS 6100

VM-FEX (VN-Link in Hardware) VN-Link in Software

Cisco

VIC as

VM-FEX


优化虚拟环境下的IO 场景 1: 软件VN-LINK 和VIC

VM VM VM VM VM

Nexus 1000V hypervisor switch

Hypervisor

Cisco Virtualized Adapter

VN-LINK in SW = Nexus 1000V

• Each VM vnic connects to Nexus 1000V hypervisor switch • Nexus 1000V switch uplinks connect to multiple distinct Cisco virtual interfaces (VIFs)

Likely Use Case:

• Customer has already standardized on Nexus 1000V • Customer deployment needs higher scalability and number of VMs

o o o


优化虚拟环境下的IO 场景 2: 硬件VN-LINK（VM-FEX))

VM-FEX (aka VN-LINK in Hardware) • Each VM vnic maps to a different virtual interface (VIF) • IO to/from VM enters Cisco hypervisor switch module and passes thru to Cisco VIF (switching not done on CPU)

Likely Use Case:

• Customer benefits from centralized Management through UCSM • Customer needs higher performance

Service Console

Kernel

dVIF53-Veth5 Profile VMK

dVIF54-Veth10 Profile COS

dVIF1

Cisco VIC adapter

VM-FEX

… dVIF2 dVIF3 dVIF4 dVIF45 dVIF46 dVIF47 dVIF48 dVIF49 dVIF50 dVIF51 dVIF52


日程



Core Layer

Aggregation Layer

Networking Services

Access Layer


Hypervisors





Virtual HBA & NPV

Unified IO & FCoE

SAN & Storage

• Q&A


网络行为和特征

• Ethernet is non-deterministic.

Flow control is destination-based

Relies on TCP drop-retransmission / sliding window

• Fibre-Channel is deterministic.

Flow control is source-based (B2B credits)

Services are fabric integrated (no loop concept)


数据中心以太网(DCB) 特性 PFC

Enables lossless Fabrics for each class of service PAUSE sent per virtual lane when buffers limit exceeded Network resources are partitioned between VL’s (E.g. input buffer and output queue) The switch behavior is negotiable per VL

Priority-based Flow Control (PFC)


DCB / FCoE 相关标准

FCoE is fully defined in the FC-BB-5 standard (since Jun/2009)

FCoE works with additional technologies to make I/O Consolidation a reality

T11 IEEE 802.1 FCoE

FC on

other

network

media

FC on

Other

Network

Media

FC-BB-5

PFC ETS DCBX

802.1Qbb

DCB

802.1Qaz 802.1Qaz

Lossless

Ethernet Priority

Grouping Configuration

Verification


FCoE is fully defined in the FC-BB-5 standard (since Jun/2009)

FCoE works with additional technologies to make I/O Consolidation a reality

T11 IEEE 802.1 FCoE

FC on

other

network

media

FC on

Other

Network

Media

FC-BB-5

PFC ETS DCBX

802.1Qbb

DCB

802.1Qaz 802.1Qaz

Lossless

Ethernet Priority

Grouping Configuration

Verification

Standard / Feature Status of the Standard

T11 BB-5

Fibre Channel over Ethernet (FCoE) Standard (Jun 3, 2009)

IEEE 802.1Qbb Priority-based Flow Control (PFC)

Forwarded to RevCom for publication in April 2011

IEEE 802.3bd Frame Format for PFC


IEEE 802.1Qaz Enhanced Transmission Selection (ETS) and Data Center Bridging eXchange protocol (DCBX)


DCB / FCoE 相关标准


FCoE Benefits

以太网承载光纤通道（FCoE）一瞥

• Mapping of FC frames over Ethernet

• Enables FC to run on a lossless Data Center Ethernet network

• Wire Server Once

• Fewer cables and adapters

• Software Provisioning of I/O

• Interoperates with existing SANs

• No gateway—stateless

• Standard – June 3, 2009

Fibre

Channel

Ethernet


统一 I/O 架构整合

Ethernet FC

LAN SAN B SAN A

No Consolidated IO I/O Consolidation with FCoE

SAN B LAN SAN A

FCoE

Nexus

5000


SA

N E

dg

e

SA

N C

ore

VSAN, NPIV, NPV, 及存储访问

SAN &

Storage


虚拟存储区域网(VSAN)

• Consolidation of SAN islands

Increased utilization of fabric ports with Just-In-Time provisioning

• Deployment of large fabrics

Dividing a large fabric in smaller VSANs

Disruptive events isolated per VSAN

RBAC for administrative tasks

Zoning is independent per VSAN

• Advanced traffic management

Defining the paths for each VSAN

VSANs may share the same EISL

Cost effective on WAN links

• Resilient SAN Extension

• Standard solution (ANSI T11 FC-FS-2 section 10)

SAN Islands

Department A

Department B Department C

Virtual SANs

(VSANs)

Department A

Department B

Department C


理解 VSANs (或 Virtual Fabrics)

Production SAN Tape SAN Test SAN

FC

FC

FC

FC

FC

FC

SAN E

DomainID=5 SAN F

Domain ID=6

FC

FC

FC

FC

SAN A

DomainID=1 SAN B

DomainID=2 SAN C

DomainID=3

SAN D

DomainID=4

DomainID=8 DomainID=7


什么是NPIV?

• N-Port ID Virtualization (NPIV) provides a means to assign multiple FC IDs to a single N port.

• This feature was intended to allow multiple applications to share the same Fiber Channel HBA

• The use of different pWWN allows access control, zoning, and port security to be implemented at the application level.

• Usage applies to applications such as Vmware vSphere, Microsoft Hyper-V and Citrix XenServer

Application Server FC Switch

Email

Web

File Services

Email I/O N_Port_ID 1

Web I/O N_Port_ID 2

File Services I/O N_Port_ID 3

F_Port


什么是NPV?

• N-Port Virtualizer (NPV) utilizes NPIV functionality to allow a ―switch‖ to act like a Server doing multiple logins through 1 physical link

• Real server connected (via CNAs) to Nexus 5x00 do not login to the Nexus 5x00 but to upstream FC switch. The same applies to FC edge switches (ex.: MDS blade switches and MDS 91xx FC fabric switches).

• No local switching is done on an FC switch in NPV mode

• FC edge switch in NPV mode Does NOT take up a Domain ID

Nexus 5x00, MDS 91xx, MDS blade switches, UCS Fabric Interconnect FC Core Switch

Eth1/1

Eth1/2

Eth1/3

Server1 N_Port_ID 1

Server2 N_Port_ID 2

Server3 N_Port_ID 3

F_Port

Server1

Server2

Server3


在FC刀片交换机上采用NPV

• Eliminates edge FC switch Domain ID

• Edge FC switch acts as an NPIV host

• Simplifies server and SAN management and operations

• Increases fabric scalability

端到端数据中心虚拟化 - cisco

Documents