vmware vsan & cisco

VMware vSAN & Cisco

John Kennedy

TME – DCG UCS group

Cisco Confidential 2© 2013-2014 Cisco and/or its affiliates. All rights reserved.

Software-Defined Storage

2

Bringing the efficient operational model of virtualization to storage

Virtual Data ServicesData

ProtectionMobility

Performance

Policy-driven Control Plane

SAN / NAS

SAN/NAS Pool

Virtual Data Plane

x86 Servers

Hypervisor-convergedStorage pool

Object Storage Pool

Cloud Object

StorageVirtual SAN


3

VMware Virtual SAN

• Software-defined storage software solution

• Aggregates locally attached storage from each ESXi host in a cluster.

• Flash optimized storage solution.

• VM-Centric data operations and policy driven management principals.

• Resilient design based on a Distributed RAID architecture

• No single points of failures

• Fully integrated with vSphere

vSphere + Virtual SAN

Hard disksHard disksSSD SSD Hard disksSSD

…

Virtual SAN Shared Datastore

• Hypervisor-Converged storage platform


4

Management Clusters

Use Cases

Backup and DR Target

DMZ / ComplianceIsolated storage

Test / Dev / StagingPrivate cloud

Virtual Desktop

ROBO

VDI

Site A Site B

vSphereVSAN


VMware vSAN technical overview


6

Virtual SAN Datastore• Virtual SAN is an object store solution that is presented to vSphere

as a file system.

• The object store mounts the VMFS volumes from all hosts in a cluster and presents them as a single shared datastore.• Only members of the cluster can access the Virtual SAN datastore

• Not all hosts need to contribute storage, but its recommended.

disk group disk group disk group disk groupEach host: 5 disk groups max. Each disk group: 1 SSD + 1 to 7

HDDs disk group

VSAN network VSAN network VSAN network VSAN networkVSAN network

vsanDatastore

HDD HDDHDDHDDHDD


7

Virtual SAN Disk Groups• Virtual SAN uses the concept of disk groups to pool together flash

devices and magnetic disks as single management constructs.

• Disk groups are composed of at least 1 flash device and 1 magnetic disk.• Flash devices are use for performance (Read cache + Write buffer).

• Magnetic disks are used for storage capacity.

• Disk groups cannot be created without a flash device.

disk group disk group disk group disk group

Each host: 5 disk groups max. Each disk group: 1 SSD + 1 to 7 HDDs disk group

HDD HDDHDDHDDHDD


8

Technical CharacteristicsVirtual SAN is a cluster level feature similar to:

• vSphere DRS

• vSphere HA

• Virtual SAN

•Deployed, configured and manage from vCenter through the vSphere Web Client (ONLY!).

• Radically simple

• Configure VMkernel interface for Virtual SAN

• Enable Virtual SAN by clicking Turn On


9

Virtual SAN Implementation Requirements

• Virtual SAN requires:

• Minimum of 3 hosts in a cluster configuration

• All 3 host MUST!!! contribute storage• vSphere 5.5 U1 or later

• Locally attached disks• Magnetic disks (HDD)

• Flash-based devices (SSD)

• Network connectivity• 1GB Ethernet

• 10GB Ethernet (preferred)

esxi-01

local storage local storage local storage

vSphere 5.5 U1 Cluster

esxi-02 esxi-03

cluster

HDDHDD HDD


10

Flash Based Devices

In Virtual SAN ALL read and write operations always go directly to the Flash tier.

Flash based devices serve two purposes in Virtual SAN

1. Non-volatile Write Buffer (30%)• Writes are acknowledged when they enter prepare stage on SSD.

• Reduces latency for writes

2. Read Cache (70%)• Cache hits reduces read latency

• Cache miss – retrieve data from HDD

Choice of hardware is the #1 performance

differentiator between Virtual SAN configurations.


11

Virtual SAN Scalable Architecture

• Scale up and Scale out architecture – granular and linearly storage, performance and compute scaling capabilities• Per magnetic disks – for capacity

• Per flash based device – for performance

• Per disk group – for performance and capacity

• Per node – for compute capacity

disk group disk group disk group

VSAN network

VSAN network

VSAN network

vsanDatastore

HDD

disk group

HDD HDD

HDD

disk group

VSAN network

HDD

scale

up

scale out


12

Storage Policy-based Management

• SPBM is a storage policy framework built into vSphere that enables virtual machine policy driven provisioning.

• Virtual SAN leverages this new framework in conjunction with VASA API’s to expose storage characteristics to vCenter:• Storage capabilities

• Underlying storage surfaces up to vCenter and what it is capable of offering.

• Virtual machine storage requirements• Requirements can only be used against available capabilities.

• VM Storage Policies• Construct that stores virtual machine’s storage provisioning requirements based on

storage capabilities.


Storage Policy Wizard

SPBM

object

object manager

virtual disk

VSAN objects may be (1) mirrored across hosts & (2) striped across disks/hosts to meet VM storage profile policies

Datastore Profile

Virtual SAN SPBM Object Provisioning Mechanism


14

Virtual SAN Capabilities

• Virtual SAN currently surfaces five unique storage capabilities to vCenter.


15

Number of Failures to Tolerate

• Number of failures to tolerate• Defines the number of hosts, disk or network failures a storage object can

tolerate. For “n” failures tolerated, “n+1” copies of the object are created and “2n+1” host contributing storage are required.

vsan network

vmdkvmdk witness

esxi-01

esxi-02 esxi-03 esxi-04

~50% of I/O ~50% of I/O

Virtual SAN Policy: “Number of failures to tolerate = 1”

raid-1


16

Virtual SAN Storage Capabilities • Number of disk stripes per object

• The number of HDDs across which each replica of a storage object is distributed. Higher values may result in better performance.

• Force provisioning

• if yes, the object will be provisioned even is the policy specified in the storage policy is not satisfiable with the resources currently available.

• Flash read cache reservation (%)

• Flash capacity reserved as read cache for the storage object. Specified as a percentage of logical size of the object.

• Object space reservation (%)

• Percentage of the logical size of the storage object that will be reserved (thick provisioned) upon VM provisioning. The rest of the storage object is thin provisioned.

•


18

Virtual SAN Delivers Enterprise-Grade ScaleMaximum Scalability per Virtual SAN Cluster

2MIOPS

3,200VMs

4.4 Petabyte

s

32Hosts

“Virtual SAN allows you to build out scalable heterogeneous storage infrastructure like the Facebook's and Google's of the world.

“Virtual SAN allows you to add scale, add resources, while being able to service high performance workloads.”


Cisco UCS with vSAN


Virtual SAN on UCS Benchmarking


Virtual SAN on UCS Benchmarking

IOPS capacity for 100% Read workload IOPS capacity for 70% Read, with 30% Write workload


SAS/SATA/PCIe Multi-level cell SSD (or better)

SAS/NL-SAS HDDSelect SATA HDDs

Any certified Server

…using the VMware Compatibility Guide* for VSAN. Hardware Combinations must be supported by the server vendor.

Enterprise-grade HBA/RAID Controller

1 2 Build your own solutionChoose a VSAN Ready Node

…categorized into performance, balanced, capacity or entry level solution profiles

Prescriptive VSAN Ready solution recommendations …

Select a pre-validated server

Two Ways to Design a Virtual SAN SolutionChoose or Build your Virtual SAN hardware platform

or

Build a component based configuration*

http://www.vmware.com/resources/compatibility/search.php?deviceCategory=vsan


UCS Starter Kit for VMware Virtual SAN

2 x Cisco UCS 6248UP 48-Port Fabric Interconnects4 x Cisco UCS C240 M3 Rack servers• CPU: 2 x 2.60-GHz Intel Xeon

processors E5-2650 v2• Memory: 8 x 16 GB (128 GB

total)• Cisco UCS Virtual Interface Card

VIC 1225• HDD: 7 x 1 TB SATA 7.2K RPM

SFF• SSD: 1 x 800 GB SAS SSD• 9271 LSI RAID controller

UCS-VSAN-IVB-28TBP

Node Expansion

1 x Cisco UCS C240 M3 Rack servers• CPU: 2 x 2.60-GHz Intel Xeon

processors E5-2650 v2• Memory: 8 x 16 GB (128 GB total)• Cisco UCS Virtual Interface Card VIC

1225• HDD: 7 x 1 TB SATA 7.2K RPM SFF• SSD: 1 x 800 GB SAS SSD• 9271 LSI RAID controller

Disk Group Expansion

• HDD: 7 x 1 TB SATA 7.2K RPM SFF• SSD: 1 x 800 GB SAS SSD

Cisco UCS Solution Accelerator Packs for VMware Virtual SAN


Cisco’s current ready nodes

Ref: http://partnerweb.vmware.com/programs/vsan/Virtual%20SAN%20Ready%20Nodes.pdf


Cisco’s current Ready Nodes



Special considerations


http://www.vmware.com/resources/compatibility/search.php?deviceCategory=vsanio&details=1&vsan_type=vsanio&io_partner=146

VMware vSAN qualified adapters from Cisco


Do not use JBOD mode…

LSI does not support 9271 in JBOD mode.

All vSAN disks must be RAID 0 individual virtual disks.

This means:

1) UCSM will not configure the disks for you.

2) You must use LSI WebCLI, or StorCLI, or MegaCLI.

3) You must configure VMware ESXi to view the SSD disks as SSDs.


Sample commands for LSI 9271 with StorCli.

#install StorCli in ESXi consoleesxcli software vib install -v=vmware-esx-storcli.vib --no-sig-check

#Delete all virtual disks on scsi ctrlstorcli /c0/vall del

#create new virtual disks in RAID0, one for each physical diskcd /opt/lsi/storcli./storcli /c0 add vd each type=raid0 pdcache=off Dimmerswitch=MaximumWithoutCaching direct wt nora NoCachedBadBBU


Telling ESXi that an SSD disk is really SSD…

Run this command to add a PSA claim rule to mark the device as SSD:

# esxcli storage nmp satp rule add --satp=SATP_TYPE --device=naa.ID --option="enable_ssd"

For example:

# esxcli storage nmp satp rule add --satp=VMW_SATP_CX --device=naa.6006016015301d00167ce6e2ddb3de11 --option="enable_ssd"

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=2013188

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=2013188


Resources


Internal mailing lists [email protected] [email protected] [email protected]

VMware HCL http://www.vmware.com/resources/compatibility/search.php?deviceCategory=vsan

VMware vSAN resource links http://www.vmware.com/products/virtual-san/resources.html

My favorite blogs http://cormachogan.com/ http://blogs.vmware.com/vsphere http://www.yellow-bricks.com/ http://www.virtuallyghetto.com/ http://bit.ly/1uLwHD8

vSAN info

mailto:[email protected]





http://cormachogan.com/

http://cormachogan.com/

http://blogs.vmware.com/vsphere

http://blogs.vmware.com/vsphere

http://www.yellow-bricks.com/

http://www.yellow-bricks.com/

http://www.virtuallyghetto.com/

http://www.virtuallyghetto.com/

http://bit.ly/1uLwHD8

http://bit.ly/1uLwHD8


Questions?”

vmware vsan & cisco

Technology